The rocket blew up because one of SpaceX's old factory ERP systems didn't sufficiently track parts, and only printed labels for frozen storage, refrigerated storage, and room temp storage.
Nobody flipped through the manifest of a fuel tank shipment to see they needed to be stored in liquid nitrogen. Someone put the shipment in frozen storage according to the label and the dry ice the tank was packed in evaporated. Cracks formed from the expansion.
Weeks or months later, someone noticed the improperly stored tank and tossed it into a tank of liquid nitrogen. The cracks shrank too small to see on the X-ray inspection. There was no inventory tracking within the freezer area.
When the fuel tank was filled with liquid oxygen, the cracks leaked, pure oxygen hit a spark, and boom.
Source: a .Net/Typescript engineer who responded to the accident by porting the quality assurance ERP system to their new ERP, Warp.
ortusdux · 4h ago
What type of tank?
Factory_Reset · 2h ago
Carbon wrapped.
vrosas · 7h ago
> Within the company, engineers and technicians actually took pressurized tanks that stored helium—one of these had burst, leading to the explosion—and shot at them in Texas to determine whether they would explode and what the result looked like.
Probably the most fun day of work in those technicians' lives.
actionfromafar · 7h ago
You could probably map the technicians' experience on a 2 dimensional grid with surreal and fun for y and x axis and allow for negative values on the fun scale.
Factory_Reset · 5h ago
The theory that it was a sniper lingered for almost 10 months.
Investigations into the catastrophic unscheduled disassembly were looking into internal failures as early as 1 week after the incident.
The 50cal used to shoot the tank was indeed fun and cool, everyone gathered around to watch the explosion.
After 10 months of investigation and destructive testing, the source of the explosion was discovered.
The trial and error necessary to find the cause led to an incredbly McGuyvered testing process/device.
Eventually this led to Space-X developing and building a new in-company supply chain for 1 specific part of the rocket that had been previously outsourced.
To my knowledge Gen 2 and Gen 3 of this part have never failed but I haven't been looking that closely till Starship trials started.
jowea · 7h ago
Huh, never realized this book[1] I read was actually distantly on a real (non-) story.
"It was, for all intents and purposes, akin to an automobile idling in a driveway with half a tank of gasoline. And then it exploded."
Not entirely unheard of.
A certain vintage of Ford cars had a cruise control defect that led them to catch fire while parked (something to do with a switch connected to the brake hydraulics that was connected to +12V from the battery even while the car was off). There were, I believe, multiple rounds of recalls (as the problem manifested itself across a number of different models).
In one case I'm familiar with, the car's owners left the car in their garage while they went on vacation. The car caught fire - fortunately the garage was sufficiently well sealed that the fire ran out of oxygen before it spread to the rest of the house.
The recall notice for their car was waiting for them at the post office when they returned home.
Damages, etc., were immediately taken care of by their insurance company and ultimately paid for by Ford - as I recall hearing the story this included thorough professional cleaning of rugs and such due to smoke from the fire making it into the rest of the house.
> > "It was, for all intents and purposes, akin to an automobile idling in a driveway with half a tank of gasoline. And then it exploded."
>
> Not entirely unheard of.
It's not even a fair analogy, the automobile is being fueled at a gas pump... which sometimes goes boom due to static discharge.
I mean, if they thought they saw a flash from the top of the competitor's building a mile away the right amount of time before the explosion, I'm not surprised they wanted to investigate the theory.
I'm also not surprised they didn't push it much publicly. It sounds like trying to deflect blame for your failures.
digdugdirk · 7h ago
It is trying to deflect blame for your failures. It also signals a lack of understanding of their own design, and an unwillingness to dig into the details to figure it out.
These are not good signals to be sending when you're trying to get a contract to send a manned mission to space.
prasadjoglekar · 7h ago
This was 9 years ago.
The article itself says the theory isn't completely kooky.
Quote:
This is not as crazy as it sounds, and other engineers at SpaceX aside from Musk entertained the possibility, as some circumstantial evidence to support the notion of an outside actor existed. Most notably, the first rupture in the rocket occurred about 200 feet above the ground, on the side of the vehicle facing the southwest. In this direction, about one mile away, lay a building leased by SpaceX's main competitor in launch, United Launch Alliance. A separate video indicated a flash on the roof of this building, now known as the Spaceflight Processing Operations Center. The timing of this flash matched the interval it would take a projectile to travel from the building to the rocket.
palmotea · 6h ago
> This is not as crazy as it sounds, and other engineers at SpaceX aside from Musk entertained the possibility, as some circumstantial evidence to support the notion of an outside actor existed. Most notably, the first rupture in the rocket occurred about 200 feet above the ground, on the side of the vehicle facing the southwest. In this direction, about one mile away, lay a building leased by SpaceX's main competitor in launch, United Launch Alliance.
It's completely kooky, so kooky in fact, there's a (IMHO very funny) sketch about turning business competition into a gunfight: https://www.youtube.com/watch?v=mpC_hO15IoA. It's funny because the idea is so absurd.
United Launch Alliance has lawyers. Blowing up your competitor's rocket with a gun would be about the stupidest thing you could possibly do. The upside is low (single rocket launch ruined), the downside is massive (huge civil and criminal penalties). It also suffers from the typical conspiracy theory weakness: everyone involved would need to keep their mouths shut forever, and realistically, at some point someone's gonna have loose lips.
marcuskane2 · 35m ago
It wouldn't need to be some plan signed off on by ULA's top brass.
It could have been a single person acting alone for whatever reason. Plausibly an employee of ULA who formerly worked at SpaceX and was mad as mistreatment there, or mad they interviewed and didn't get hired, or mad that SpaceX was beating them at whatever and making them look bad or whatever.
Or it could have been a person unaffiliated with ULA who just found a way to access their roof.
History has a handful of pretty significant events that were basically "one guy with a rifle".
oskarkk · 2h ago
> Blowing up your competitor's rocket with a gun would be about the stupidest thing you could possibly do. The upside is low (single rocket launch ruined)
Not commenting on the overall viability of shooting a rocket by competition (it would be stupid IMO), but it's not just a single launch ruined. Launch failure can stop launches/production for months (3-4 months in this case, 5 in the case of the previous SpaceX failure), or even longer if finding the cause of the failure is very hard. A pause in business like that can destroy a company that needs cash. Also, it reduces customers' confidence in the failing rocket, which would be good for the competitor with a flawless record.
Compare that with Boeing's MCAS problems, the plane crashes themselves were less financially damaging than the resulting groundings, production pauses and canceled orders.
squigz · 6h ago
> The timing of this flash matched the interval it would take a projectile to travel from the building to the rocket.
This seems a wildly broad description, doesn't it? What interval, for what projectile?
AnimalMuppet · 6h ago
Well, projectiles would travel somewhere between just under mach 1 and a top limit of... mach 3, maybe? So the flash would have to be some time between 1.5 and 5 seconds before the explosion? That is not a very large window.
tekla · 6h ago
Why would you even bother to bring this up. This is a non-technical consumer oriented news report, not an engineering analysis.
SpaceX saw a weird coincidence that might be worth investigating. Thats all you need to know.
GuB-42 · 6h ago
It is not unwillingness to dig into details to consider that the rocket may have been shot in addition to the more mundane explanations. And they finally found the root cause, published their findings, and fixed the problem, nothing out of the ordinary.
Of course they didn't understand what happened right away, in fact, it would have been worrying if they did. What did you expect? Something like: "Oh yeah, it was a leak, it happens sometimes, we just hoped that it wouldn't go boom"?
The focus on the sniper theory is just one of Elon Musk's ramblings. Good thing SpaceX is not just Elon Musk.
panick21_ · 6h ago
> unwillingness to dig into the details to figure it out
They did that too. They didn't only research in one direction.
The reality is that this was a incredibly unique thing that had never been seen before. So it took longer then most investigations.
actionfromafar · 7h ago
Wouldn't leaving a muzzle flash be almost comically incompetent?
AnimalMuppet · 7h ago
I don't know that much about sniper rifles. Can you not leave a muzzle flash? Is that possible (with flashless powder or something)? And if you do, what are the trade-offs?
But a flash in the middle of the day would not be very visible anyway.
free_bip · 7h ago
Silencers can reduce or even entirely eliminate muzzle flash.
lazide · 7h ago
Yup. Definitely won’t suppress the supersonic crack from any projectile able to deliver useful energy at that range though.
reverendsteveii · 23m ago
wasn't there a rocket launch going on at the time the shot was fired?
coldburst · 6h ago
Suppressors only reduce the loudness of the muzzle blast without impacting the projectile's ability to break the sonic barrier. In fact, suppressors slightly increase projectile speed because they effectively add barrel length.
lazide · 6h ago
First part - that’s what I said?
Second part - no suppressor I’ve ever seen or used did that, they all did the opposite.
taylodl · 7h ago
Sniper rifles have longer barrels that both suppress barrel flash and increase muzzle velocity - thus making the rifle more accurate at longer distances. They also fire larger rounds with larger cartridges to get even greater muzzle velocity. The trade-off is size and weight. Longer barrels, larger rounds, bigger cartridges - that increases the size and weight of the weapon and its ammunition. That's size and weight an infantry neither needs nor wants.
thijson · 7h ago
The Boer's during the Boer war were able to using a sniping attack against the British because they had smokeless gunpowder, so I would imagine flashless is also an advantage. I remember seeing artillery and tanks having something on the end of the barrel that diverts the flame to the sides.
mmh0000 · 7h ago
The thing on a tank is called a “muzzle brake” and it greatly reduces felt recoil. Also available on many smaller guns.
I mean it's pretty easy to hide the flash. Said flash is just burning powder after it leaves the barrel. So if you have an expansion chamber after the barrel it will be contained. Silencers do this secondarily with the first order effect of reducing the noise. An empty silencer (no baffles) will still hide the majority if not all the flash.
lazide · 7h ago
Also, flash hiders are quite effective.
actionfromafar · 7h ago
It's not very noticeable in daylight from a distance in the first place, plus you would shoot from inside a room which would all but completely hide it.
(The whole idea seems a bit crazy too, there are a thousand things which can go wrong with a rocket, so let's look for snipers.)
mingus88 · 7h ago
This is the first time I’ve heard of this incident and I’m not ashamed to admit that with everything we now know about Musk, it seems totally plausible that he would have internally squashed any other theories.
He has convinced his shareholders that FSD will ship by the end of every year for the past decade, so why wouldn’t he believe he could convince the FBI that the only way his super fueling process could have gone wrong is via an action movie sniper plot?
AnimalMuppet · 7h ago
I think you're going a bit too far. The article doesn't say that he internally squashed any other theories. Their investigation successfully found the real issue.
lenerdenator · 7h ago
> (The whole idea seems a bit crazy too, there are a thousand things which can go wrong with a rocket, so let's look for snipers.)
If I didn't know any better, I'd say some guy who both roasted his brain with drugs and spends a lot of time in the company of a person who was sniped at is in charge of the company.
anton-c · 7h ago
I mean a competitor shooting from their own property is just comical corporate sabotage too considering we're dealing with rocket scientists here.
tekla · 6h ago
There are a thousand things that can go wrong when the Rocket is doing things.
It exploding while just sitting there doing nothing narrows the possibilities 99.9%.
actionfromafar · 6h ago
Things happen while sitting, fuelling etc. But this rocket was in the air when it exploded IIRC.
tekla · 6h ago
What? Amos-6 blew up on the launch pad. The article headline picture shows it on the pad.
jajko · 7h ago
If you want to shoot 1 mile ammo, you need anti-material stuff like .50 BMG or at least .338 with some very hot loads. Probably even bigger if you want to have better chance, wind can easily carry the round 10 meters aside even in calmer conditions. Plus bullet would leave very distinct entry and exit points that couldn't be mismatched with some random shrapnel or other damage, or get lodged in more solid parts of rocket body and it would still be identifiable as bullet.
Such shot would be heard for a mile in quieter conditions. Not joking. Even during distant launch, a perimeter of few 100s of meters of shooter anybody there would hear it clearly.
I never owned or operated seriously any gun, just followed few gun channels + you have all the info on wikis and elsewhere, its just rather basic physics.
Anybody with similar or better knowledge would quickly debunk such theory as bs and focus more on known facts (but we talk about elon's ego here, not the most rational entity, he ie prefers pedophilia accusations when things don't roll as he wants). If they stated some very powerful laser gun, mkay more reasonable but then no visible flashes outside target itself. And such gun would require massive installation, preparation, testing etc.
aeonik · 6h ago
You could reasonably hit a target within 36 inches at a mile with a lot of different calibers.
You don't necessarily need large caliber. .50 cal has a great ballistic coefficient, but bullet design and speed matters a lot too.
6.5 creedmoor and others in the intermediate precision class can all do this as well.
That being said 1 mile is really really long shot, and right where a lot of the lighter calibers start getting near transonic.
Wind is another beast, if it's steady it can be compensated for, but I don't know what the weather was that day... chaotic gusts will wreck a shot for sure.
All that being said, the rocket is a pretty big target, 12 feet across and almost 230 feet high... that's going to compensate for a lot.
Agreed that if you found the piece where the bullet punctured it should be pretty evident what happened. But I don't have any experience doing investigations in this area, and I've heard at least from plane crashes that the fuselage can disintegrate pretty readily in high force scenarios.
I have operated and owned precision rifles, but not big bore, and have not gone out to extreme distances. More fun to play with the math and ballistics.
Would love to hear from someone who knows more than either of us.
7e · 6h ago
This is Elon Musk paranoia being pushed onto a bunch of n00b kids pretending to be engineers.
JohnTHaller · 4h ago
[flagged]
Factory_Reset · 2h ago
Why would they.
The findings of that investigation lead to a massive improvement in materials engineering for their rockets.
Nobody flipped through the manifest of a fuel tank shipment to see they needed to be stored in liquid nitrogen. Someone put the shipment in frozen storage according to the label and the dry ice the tank was packed in evaporated. Cracks formed from the expansion.
Weeks or months later, someone noticed the improperly stored tank and tossed it into a tank of liquid nitrogen. The cracks shrank too small to see on the X-ray inspection. There was no inventory tracking within the freezer area.
When the fuel tank was filled with liquid oxygen, the cracks leaked, pure oxygen hit a spark, and boom.
Source: a .Net/Typescript engineer who responded to the accident by porting the quality assurance ERP system to their new ERP, Warp.
Probably the most fun day of work in those technicians' lives.
Investigations into the catastrophic unscheduled disassembly were looking into internal failures as early as 1 week after the incident.
The 50cal used to shoot the tank was indeed fun and cool, everyone gathered around to watch the explosion.
After 10 months of investigation and destructive testing, the source of the explosion was discovered.
The trial and error necessary to find the cause led to an incredbly McGuyvered testing process/device.
Eventually this led to Space-X developing and building a new in-company supply chain for 1 specific part of the rocket that had been previously outsourced.
To my knowledge Gen 2 and Gen 3 of this part have never failed but I haven't been looking that closely till Starship trials started.
[1] https://www.goodreads.com/book/show/128591386-the-eighth-con...
Not entirely unheard of.
A certain vintage of Ford cars had a cruise control defect that led them to catch fire while parked (something to do with a switch connected to the brake hydraulics that was connected to +12V from the battery even while the car was off). There were, I believe, multiple rounds of recalls (as the problem manifested itself across a number of different models).
In one case I'm familiar with, the car's owners left the car in their garage while they went on vacation. The car caught fire - fortunately the garage was sufficiently well sealed that the fire ran out of oxygen before it spread to the rest of the house.
The recall notice for their car was waiting for them at the post office when they returned home.
Damages, etc., were immediately taken care of by their insurance company and ultimately paid for by Ford - as I recall hearing the story this included thorough professional cleaning of rugs and such due to smoke from the fire making it into the rest of the house.
I'm also not surprised they didn't push it much publicly. It sounds like trying to deflect blame for your failures.
These are not good signals to be sending when you're trying to get a contract to send a manned mission to space.
The article itself says the theory isn't completely kooky. Quote: This is not as crazy as it sounds, and other engineers at SpaceX aside from Musk entertained the possibility, as some circumstantial evidence to support the notion of an outside actor existed. Most notably, the first rupture in the rocket occurred about 200 feet above the ground, on the side of the vehicle facing the southwest. In this direction, about one mile away, lay a building leased by SpaceX's main competitor in launch, United Launch Alliance. A separate video indicated a flash on the roof of this building, now known as the Spaceflight Processing Operations Center. The timing of this flash matched the interval it would take a projectile to travel from the building to the rocket.
It's completely kooky, so kooky in fact, there's a (IMHO very funny) sketch about turning business competition into a gunfight: https://www.youtube.com/watch?v=mpC_hO15IoA. It's funny because the idea is so absurd.
United Launch Alliance has lawyers. Blowing up your competitor's rocket with a gun would be about the stupidest thing you could possibly do. The upside is low (single rocket launch ruined), the downside is massive (huge civil and criminal penalties). It also suffers from the typical conspiracy theory weakness: everyone involved would need to keep their mouths shut forever, and realistically, at some point someone's gonna have loose lips.
It could have been a single person acting alone for whatever reason. Plausibly an employee of ULA who formerly worked at SpaceX and was mad as mistreatment there, or mad they interviewed and didn't get hired, or mad that SpaceX was beating them at whatever and making them look bad or whatever.
Or it could have been a person unaffiliated with ULA who just found a way to access their roof.
History has a handful of pretty significant events that were basically "one guy with a rifle".
Not commenting on the overall viability of shooting a rocket by competition (it would be stupid IMO), but it's not just a single launch ruined. Launch failure can stop launches/production for months (3-4 months in this case, 5 in the case of the previous SpaceX failure), or even longer if finding the cause of the failure is very hard. A pause in business like that can destroy a company that needs cash. Also, it reduces customers' confidence in the failing rocket, which would be good for the competitor with a flawless record.
Compare that with Boeing's MCAS problems, the plane crashes themselves were less financially damaging than the resulting groundings, production pauses and canceled orders.
This seems a wildly broad description, doesn't it? What interval, for what projectile?
SpaceX saw a weird coincidence that might be worth investigating. Thats all you need to know.
Of course they didn't understand what happened right away, in fact, it would have been worrying if they did. What did you expect? Something like: "Oh yeah, it was a leak, it happens sometimes, we just hoped that it wouldn't go boom"?
The focus on the sniper theory is just one of Elon Musk's ramblings. Good thing SpaceX is not just Elon Musk.
They did that too. They didn't only research in one direction.
The reality is that this was a incredibly unique thing that had never been seen before. So it took longer then most investigations.
But a flash in the middle of the day would not be very visible anyway.
Second part - no suppressor I’ve ever seen or used did that, they all did the opposite.
https://en.m.wikipedia.org/wiki/Muzzle_brake
(The whole idea seems a bit crazy too, there are a thousand things which can go wrong with a rocket, so let's look for snipers.)
He has convinced his shareholders that FSD will ship by the end of every year for the past decade, so why wouldn’t he believe he could convince the FBI that the only way his super fueling process could have gone wrong is via an action movie sniper plot?
If I didn't know any better, I'd say some guy who both roasted his brain with drugs and spends a lot of time in the company of a person who was sniped at is in charge of the company.
It exploding while just sitting there doing nothing narrows the possibilities 99.9%.
Such shot would be heard for a mile in quieter conditions. Not joking. Even during distant launch, a perimeter of few 100s of meters of shooter anybody there would hear it clearly.
I never owned or operated seriously any gun, just followed few gun channels + you have all the info on wikis and elsewhere, its just rather basic physics.
Anybody with similar or better knowledge would quickly debunk such theory as bs and focus more on known facts (but we talk about elon's ego here, not the most rational entity, he ie prefers pedophilia accusations when things don't roll as he wants). If they stated some very powerful laser gun, mkay more reasonable but then no visible flashes outside target itself. And such gun would require massive installation, preparation, testing etc.
You don't necessarily need large caliber. .50 cal has a great ballistic coefficient, but bullet design and speed matters a lot too.
6.5 creedmoor and others in the intermediate precision class can all do this as well.
That being said 1 mile is really really long shot, and right where a lot of the lighter calibers start getting near transonic.
Wind is another beast, if it's steady it can be compensated for, but I don't know what the weather was that day... chaotic gusts will wreck a shot for sure.
All that being said, the rocket is a pretty big target, 12 feet across and almost 230 feet high... that's going to compensate for a lot.
Agreed that if you found the piece where the bullet punctured it should be pretty evident what happened. But I don't have any experience doing investigations in this area, and I've heard at least from plane crashes that the fuselage can disintegrate pretty readily in high force scenarios.
I have operated and owned precision rifles, but not big bore, and have not gone out to extreme distances. More fun to play with the math and ballistics.
Would love to hear from someone who knows more than either of us.