If there's anything that needs audio/video automation, I've always turned to FFmpeg, it's such a crucial and indispensible tool and so many online video tools use it and are generally a UI wrapper around this wonderful tool. TIL - there's FFmpeg.Wasm also [0].
In Jan 2024, I had used it to extract frames of 1993 anime movie in 15 minutes video segments, upscaled it using Real-ESRGAN-ncnn-vulkan [1] then recombining the output frames for final 4K upscaled anime [2]. FWIW, if I had built a UI on this workflow it could've become a tool similar to Topaz AI which is quite popular these days.
Happy to hear that they've introduced video encoders and decoders based on compute shaders. The only video codecs widely supported in hardware are H.264, H.265 and AV1, so cross-platform acceleration for other codecs will be very nice to have, even if it's less efficient than fixed-function hardware. The new ProRes encoder already looks useful for a project I'm working on.
> Only codecs specifically designed for parallelised decoding can be implemented in such a way, with more mainstream codecs not being planned for support.
It makes sense that most video codecs aren't amenable to compute shader decoding. You need tens of thousands of threads to keep a GPU busy, and you'll struggle to get that much parallelism when you have data dependencies between frames and between tiles in the same frame.
I haven't even had a cursory look at decoders state of the art for 10+ years. But my intuition would say that decoding for display could profit a lot from GPU acceleration for later parts of the process when there is already pixel data of some sort involved. Then I imagine thet the initial decompression steps could stay on the CPU and the decompressed, but still (partially) encoded data is streamed to the GPU for the final transformation steps and application to whatever I-frames and other base images there are. Steps like applying motion vectors, iDCT... look embarrassingly parallel at a pixel level to me.
When the resulting frame is already in a GPU texture then, displaying it has fairly low overhead.
My question is: how wrong am I?
happymellon · 12m ago
> Happy to hear that they've introduced video encoders and decoders based on compute shaders.
This is great news. I remember being laughed at when I initially asked whether the Vulkan enc/dec were generic because at the time it was all just standardising interfaces for the in-silicon acceleration.
Having these sorts of improvements available for legacy hardware is brilliant, and hopefully a first route that we can use to introduce new codecs and improve everyone's QOL.
mtillman · 1h ago
Exciting! I am consistently blown away by the talent of the ffmpeg maintainers. This is fairly hard stuff in my opinion and they do it for free.
droopyEyelids · 1h ago
Could you explain more about it? I assumed the maintainers are doing it as part of their jobs for a company (completely baseless assumption)
refulgentis · 45m ago
Reupvoted you from gray because I don't think that's fair, but I also don't know how much there is to add. As far as why I'm contributing, I haven't been socially involved in the ffmpeg dev community in a decade, but, it is a very reasonable floor to assume it's 80% not full time paid contributors.
pmarreck · 3h ago
Impressed anytime I have to use it (even if I have to study its man page again or use an LLM to construct the right incantation or use a GUI that just builds the incantation based on visual options). Becoming an indispensable transcoding multitool.
I think building some processing off of Vulkan 1.3 was the right move. (Aside, I also just noticed yesterday that Asahi Linux on Mac supports that standard as well.)
Culonavirus · 3h ago
> incantation
FFmpeg arguments, the original prompt engineering
agos · 1h ago
OT, but yours has to be the best username on this site. Props.
bobsmooth · 45m ago
Culón is Spanish for big-bottomed, for anyone else wondering.
mrandish · 2h ago
I'd also include Regex in the list of dark arts incantations.
RedShift1 · 1h ago
I'm ok with regex, but the ffmpeg manpage, it scares me...
Keyframe · 2h ago
with gemini-cli and claude-cli you can now prompt while it prompts ffmpeg, and it does work.
conradev · 2h ago
Yeah, you can give an LLM queries like “make this smaller with libx265 and add the hvc1 tag” or “concatenate these two videos” and it usually crushes it. They have a similar level of mastery over imagemagick, too!
turnsout · 1h ago
Yeah, LLMs have honestly made ffmpeg usable for me, for the first time. The difficulty in constructing commands is not really ffmpeg's fault—it's just an artifact of the power of the tool and the difficulties in shoehorning that power into flags for a single CLI tool. It's just not the ideal human interface to access ffmpeg's functionality. But keeping it CLI makes it much more useful as part of a larger and often automated workflow.
profsummergig · 1h ago
Just seeking a clarification on how this would be done:
One would use gemini-cli (or claude-cli),
- and give a natural language prompt to gemini (or claude) on what processing needs to be done,
- with the correct paths to FFmpeg and the media file,
- and g-cli (or c-cli) would take it from there.
Is this correct?
RedShift1 · 1h ago
Yes. It works amazingly well for ffmpeg.
profsummergig · 1h ago
Thank you.
NSUserDefaults · 2h ago
Curious to see how quickly each LLM picks up the new codecs/options.
stevejb · 2h ago
I use the Warp terminal and I can ask it to run —-help and it figures it out
baq · 2h ago
the canonical (if that's the right word for a 2-year-old technique) solution is to paste the whole manual into the context before asking questions
fwiw, `tar xzf foobar.tgz` = "_x_tract _z_e _f_iles!" has been burned into my brain. It's "extract the files" spoken in a Dr. Strangelove German accent
Better still, I recently discovered `dtrx` (https://github.com/dtrx-py/dtrx) and it's great if you have the ability to install it on the host. It calls the right commands and also always extracts into a subdir, so no more tar-bombs.
If you want to create a tar, I'm sorry but you're on your own.
diggan · 1h ago
I used tar/unzip for decades I think, before moving to 7z which handles all formats I throw at it, and have the same switch for when you want to decompress into a specific directory, instead of having to remember which one of tar and unzip uses -d, and which one uses -C.
"also always extracts into a subdir" sounds like a nice feature though, thanks for sharing another alternative!
fullstop · 1h ago
I have so much of tar memorized. cpio is super funky to me, though.
porridgeraisin · 2h ago
Personally I never understood the problem with tar balls.
The only options you ever need are tar -x, tar -c (x for extract and c for create). tar -l if you wanna list, l for list.
That's really it, -v for verbose just like every other tool if you wish.
Examples:
tar -c project | gzip > backup.tar.gz
cat backup.tar.gz | gunzip | tar -l
cat backup.tar.gz | gunzip | tar -x
You never need anything else for the 99% case.
BeepInABox · 1h ago
For anyone curious, unless you are running a 'tar' binary from the stone ages, just skip the gunzip and cat invocations. Replace .gz with .xz or other well known file ending for different compression.
Examples:
tar -cf archive.tar.gz foo bar # Create archive.tar.gz from files foo and bar.
tar -tvf archive.tar.gz # List all files in archive.tar.gz verbosely.
tar -xf archive.tar.gz # Extract all files from archive.tar.gz
tombert · 2h ago
Yeah I never really understood why people complain about tar; 99% of what you need from it is just `tar -xvf blah.tar.gz`.
aidenn0 · 2h ago
You for got the -z (or -a with a recent gnutar).
adastra22 · 2h ago
It’s no longer needed. You can leave it out and it auto-detects the file format.
CamperBob2 · 1h ago
What value does tar add over plain old zip? That's what annoys me about .tar files full of .gzs or .zips (or vice versa) -- why do people nest container formats for no reason at all?
I don't use tape, so I don't need a tape archive format.
diggernet · 1h ago
A tar of gzip or zip files doesn't make sense. But gzipping or zipping a tar does.
Gzip only compresses a single file, so .tar.gz lets you bundle multiple files.
You can do the same thing with zip, of course, but...
Zip compresses individual files separately in the container, ignoring redundancies between files. But .tar.gz (and .tar.zip, though I've rarely seen that combination) bundles the files together and then compresses them, so can get better compression than .zip alone.
fullstop · 1h ago
zip doesn't retain file ownership or permissions.
diggernet · 1h ago
Good point. And if I remember right, tar allows longer paths than zip.
sdfsdfgsdgg · 2h ago
> tar -l if you wanna list, l for list.
Surely you mean -t if you wanna list, t for lisT.
l is for check-Links.
-l, --check-links
(c and r modes only) Issue a warning message unless all links to each file are archived.
And you don't need to uncompress separately. tar will detect the correct compression algorithm and decompress on its own. No need for that gunzip intermediate step.
porridgeraisin · 2h ago
> -l
Whoops, lol.
> on its own
Yes.. I'm aware, but that's more options, unnecessary too, just compose tools.
sdfsdfgsdgg · 2h ago
That's the thing. It’s not more options. During extraction it picks the right algorithm automatically, without you needing to pass another option.
drivers99 · 2h ago
Except it's tar -t to list, not -l
porridgeraisin · 2h ago
Whoops, lol. Well that's unfortunate.
bigstrat2003 · 2h ago
The problem is it's very non-obvious and thus is unnecessarily hard to learn. Yes, once you learn the incantations they will serve you forever. But sit a newbie down in front of a shell and ask them to extract a file, and they struggle because the interface is unnecessarily hard to learn.
encom · 2h ago
It's very similar to every other CLI program, I really don't understand what kind of usability issue you're implying is unique to tar?
mrguyorama · 1h ago
As has been clearly demonstrated in this very thread, why is "Please list what files are in this archive" the option "-t"?
Principle of least surprise and all that.
jeanlucas · 2h ago
it was just a reference to xkcd#1168
I wasn't expecting the downvotes for an xkcd reference
sho_hn · 2h ago
nope, it's using `find`.
agys · 2h ago
LLMs and complex command line tools like FFmpeg and ImageMagick are a perfect combination and work like magic…
It’s really the dream UI/UX from sience fiction movies: “take all images from this folder and crop 100px away except on top, saturate a bit and save them as uncompressed tiffs in this new folder, also assemble them in a video loop, encode for web”.
xandrius · 2h ago
Had to do exactly that with a bunch of screenshots I took but happened to include a bunch of unnecessary parts of the screen.
A prompt to ChatGPT and a command later and all were nicely cropped in a second.
The dread of doing it by hand and having it magically there a minute later is absolutely mind blowing. Even just 5 years ago, I would have just done it manually as it would have definitely taken more to write the code for this task.
Barrin92 · 2h ago
it can work but it's far from science fiction. LLMs tend to produce extremely subpar if not buggy ffmpeg code. They'll routinely do things like put the file parameter before the start time which needlessly decodes the entire video, produce wrong bitrates, re-encode audio needlessly, and so on.
If you don't care enough about potential side effects to read the manual it's fine, but a dream UX it is not because I'd argue that includes correctness.
euroderf · 54m ago
Are you accusing Blade Runner of infringing FFmpeg IP ?
jjcm · 2h ago
LLMs are a great interface for ffmpeg. There are tons of tools out there that can help you run it with natural language. Here's my personal script: https://github.com/jjcm/llmpeg
Has anyone made a good GUI frontend for accessing the various features of FFMPEG? Sometimes you just want to remux a video without doing any transcoding, or join several video and audio streams together (same codecs).
filmgirlcw · 53m ago
For Mac users, ffWorks [1] is an amazing frontend for FFmpeg that surfaces most of the features but with a decent GUI. It’s batchable and you can setup presets too. It’s one of my favorite apps and the developer is very responsive.
Handbrake and Losslssscut are great too. But in addition to donating to FFmpeg, I pay for ffWorks because it really does offer a lot of value to me. I don’t think there is anything close to its polish on other platforms, unfortunately.
It's a great tool. Little long in the tooth these days, but gets the job done.
kevinsync · 1h ago
Seconded, HandBrake[0] is great for routine tasks / workflows. The UI could be simplified just a tad for super duper simple stuff (ex. ripping a multi-episode tv show disc but don't care about disc extras? you kind of have to hunt and poke based on stream length to decide which parts are the actual episodes. The app itself could probably reliably guess and present you with a 1-click 'queue these up' flow for instance) but otherwise really a wonderful tool!
I have found the best front-end to be ChatGPT. It is very good at figuring out the commands needed to accomplish something in FFmpeg, from my natural description of what I want to do.
I haven't used a GUI I like, but LLMs like ChatGPT have been so good for solving this for me. I tell it exactly what I need it to do and it produces the ffmpeg command to do it.
onehair · 1h ago
There is handbrake, vidcoder and all sorts of frontend.
ricardojoaoreis · 2h ago
You can use mkvtoolnix for that and it has a GUI
mrguyorama · 1h ago
Shotcut is an open source Video production toolkit that is basically just a really nice interface for generating ffmpeg commands.
Pretty sure ChatGPT counts as a CLI, not as a GUI.
AlienRobot · 1h ago
It would need to be a non-linear editor node-based editor. Pretty much all open source video editors are just FFMPEG frontends, e.g. Kdenlive.
larodi · 2h ago
Is anyone else on the opinion that ffmpeg now ranks 4th as the most used lib after ssl, zlib, and sqlite... given video is like omnipresent in 2025?
PokestarFan · 3m ago
FFMpeg is probably not as up high since video processing only needs to be done on the servers that receive media. I doubt most phones are running FFMpeg on video.
It's up there in the hall of fame, that's for sure!
pledg · 2h ago
libcurl?
encom · 2h ago
libc :D
joshuat · 2h ago
Some Netflix devs are going to have a busy sprint
elektor · 2h ago
For those out of the loop, can you please explain your comment?
henryfjordan · 2h ago
Netflix uses FFMPEG, will have to update
TeeMassive · 1h ago
And some influencers ;)
oldgregg · 2h ago
LLMs have really made ffmpeg implementations easy-- the command line options are so expansive and obscure it's so nice to just tell it what you want and have it spit out a crazy ffmpeg command.
instagraham · 1h ago
I remember saving my incantation to download and convert a youtube playlist (in the form of a txt file with a list of URLs) and this being the only way to back up Chrome music bookmark folders.
Then it stopped working until I updated youtube-dl and then that stopped working once I lost the incantation :<
noman-land · 1h ago
Check out yt-dlp. It works great.
TeeMassive · 58m ago
yt-dlp works really well, and not only for YouTube ;)
ok123456 · 1h ago
Finally! RealVideo 6 support.
javier2 · 50m ago
What is the performance like for AV1 / h264 in vulkan vs not vulkan?
JadoJodo · 3h ago
I don't know a huge amount about video encoding, but I presume this is one of those libraries outlined in xkcd 2347[0]?
Yeah, basically anytime a video or audio is being recorded, played, or streamed its from ffmpeg. It runs on a couple planets [0], and on most devices (maybe?)
FFMpeg is definitely fairly ubiquitous, but you are overstating its universality quite a bit. There are alternatives that utilize Windows/macOS's native media frameworks, proprietary software that utilizes bespoke frameworks, and libraries that function independently of ffmpeg that offer similar functionality.
That being said, if you put down a pie chart of media frameworks (especially for transcoding or muxing), ffmpeg would have a significant share of that pie.
neckro23 · 2h ago
Not necessarily. A lot of video software either leverages the Windows/MacOS system codecs (ex. Media Player Classic, Quicktime) or proprietary vendor codecs (Adobe/Blackmagic).
Linux doesn't really have a system codec API though so any Linux video software you see (ex. VLC, Handbrake) is almost certainly using ffmpeg under the hood (or its foundation, libavcodec).
aidenn0 · 2h ago
Pretty much.
It also was originally authored by the same person who did lzexe, tcc, qemu, and the current leader for the large text compression benchmark.
Oh, and for most of the 2010's there was a fork due to interpersonal issues on the team.
tombert · 2h ago
Yeah I think pretty much everything that involves video on Linux or FreeBSD in 2025 involves FFmpeg or Gstreamer, usually the former.
It’s exceedingly good software though, and to be fair I think it’s gotten a fair bit of sponsorship and corporate support.
_kb · 1h ago
It's the big flat one at the bottom.
zhengyi13 · 2h ago
Yes, this is a pretty fundamental building block; just not so rickety.
This seemed to be interesting to users of this site. tl;dr they added support for whisper, an OpenAI model for speech-to-text, which should allow autogeneration of captions via ffmpeg
bachittle · 1h ago
Heads up: Whisper support depends on how your FFmpeg was built. Some packages will not include it yet. Check with `ffmpeg -buildconf` or `ffmpeg -filters | grep whisper`. If you compile yourself, remember to pass `--enable-whisper` and give the filter a real model path.
Culonavirus · 2h ago
these days most movies and series already come out with captions, but you know what does not, given the vast amount of it?... ;)
yep, finally the deaf will able to read what people are saying in a porno!
0xbeefcab · 2h ago
True, but also it can be hard to find captions in languages besides english for some lesser known movies/shows
yieldcrv · 2h ago
And also pirated releases are super weird and all over the place with subtitles and video player compatibility
This could streamline things
bobsmooth · 41m ago
There's websites where you can download subtitles. Usually from very obviously pirated released.
shmerl · 51m ago
Nice! Looking forward to try WHIP/WebRTC based streaming to replace SRT.
Sean-Der · 41m ago
What are you using WHIP against today?
I am curious about adoption and features that would make big difference to users :)
josteink · 2h ago
Nice! Anyone have any idea how and when this will affect downstream projects like yt-dlp, jellyfin, etc? Especially with regard to support for HW-acceleration?
waihtis · 1h ago
T3.gg in shambles
ekianjo · 2h ago
Vulkan based encoders and decoders are super exciting!
oblio · 3h ago
First of all: congratulations!!!
Secondly, just curious: any insiders here?
What changed? I see the infrastructure has been upgraded, this seems like a big release, etc. I guess there was a recent influx of contributors? A corporate donation? Something else?
exprez135 · 3h ago
Not an insider, but I noticed that there is now a filter for using Whisper (C++) for audio transcription [1]. It looks like you provide the path to a model file [2].
This is big news if it means realtime subtitle generation.
ranger_danger · 2h ago
in my experience whisper (at least on my 3070 Ti) is not capable of high quality real-time transcription. A few seconds per second of audio, maybe.
y_sellami · 2h ago
about time vulkan got into the game.
brcmthrowaway · 2h ago
How much ARM acceleration vs x8664?
zzzeek · 1h ago
ffmpeg is a treasure to the open source and audio technology communities. The tool cuts right through all kinds of proprietary and arcane roadblocks presented by various codecs and formats and it's clear a tremendous amount of work goes into keeping it all working. The CLI is of course quite opaque and the documentation for various features is often terse, but it's still the only tool on any platform anywhere that will always get you what you need for video and audio processing without ever running up against some kind of commercial paywall.
jeanlucas · 2h ago
cheers for one more release, hope it gets attention and necessary funding
If there's anything that needs audio/video automation, I've always turned to FFmpeg, it's such a crucial and indispensible tool and so many online video tools use it and are generally a UI wrapper around this wonderful tool. TIL - there's FFmpeg.Wasm also [0].
In Jan 2024, I had used it to extract frames of 1993 anime movie in 15 minutes video segments, upscaled it using Real-ESRGAN-ncnn-vulkan [1] then recombining the output frames for final 4K upscaled anime [2]. FWIW, if I had built a UI on this workflow it could've become a tool similar to Topaz AI which is quite popular these days.
[0]: https://github.com/ffmpegwasm/ffmpeg.wasm
[1]: https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan
[2]: https://files.horizon.pics/3f6a47d0-429f-4024-a5e0-e85ceb0f6...
> Only codecs specifically designed for parallelised decoding can be implemented in such a way, with more mainstream codecs not being planned for support.
It makes sense that most video codecs aren't amenable to compute shader decoding. You need tens of thousands of threads to keep a GPU busy, and you'll struggle to get that much parallelism when you have data dependencies between frames and between tiles in the same frame.
I wonder whether encoders might have more flexibility than decoders. Using compute shaders to encode something like VP9 (https://blogs.gnome.org/rbultje/2016/12/13/overview-of-the-v...) would be an interesting challenge.
When the resulting frame is already in a GPU texture then, displaying it has fairly low overhead.
My question is: how wrong am I?
This is great news. I remember being laughed at when I initially asked whether the Vulkan enc/dec were generic because at the time it was all just standardising interfaces for the in-silicon acceleration.
Having these sorts of improvements available for legacy hardware is brilliant, and hopefully a first route that we can use to introduce new codecs and improve everyone's QOL.
I think building some processing off of Vulkan 1.3 was the right move. (Aside, I also just noticed yesterday that Asahi Linux on Mac supports that standard as well.)
FFmpeg arguments, the original prompt engineering
One would use gemini-cli (or claude-cli),
- and give a natural language prompt to gemini (or claude) on what processing needs to be done,
- with the correct paths to FFmpeg and the media file,
- and g-cli (or c-cli) would take it from there.
Is this correct?
ffmpeg right after
fwiw, `tar xzf foobar.tgz` = "_x_tract _z_e _f_iles!" has been burned into my brain. It's "extract the files" spoken in a Dr. Strangelove German accent
Better still, I recently discovered `dtrx` (https://github.com/dtrx-py/dtrx) and it's great if you have the ability to install it on the host. It calls the right commands and also always extracts into a subdir, so no more tar-bombs.
If you want to create a tar, I'm sorry but you're on your own.
"also always extracts into a subdir" sounds like a nice feature though, thanks for sharing another alternative!
The only options you ever need are tar -x, tar -c (x for extract and c for create). tar -l if you wanna list, l for list.
That's really it, -v for verbose just like every other tool if you wish.
Examples:
You never need anything else for the 99% case.I don't use tape, so I don't need a tape archive format.
Gzip only compresses a single file, so .tar.gz lets you bundle multiple files. You can do the same thing with zip, of course, but...
Zip compresses individual files separately in the container, ignoring redundancies between files. But .tar.gz (and .tar.zip, though I've rarely seen that combination) bundles the files together and then compresses them, so can get better compression than .zip alone.
Surely you mean -t if you wanna list, t for lisT.
l is for check-Links.
And you don't need to uncompress separately. tar will detect the correct compression algorithm and decompress on its own. No need for that gunzip intermediate step.Whoops, lol.
> on its own
Yes.. I'm aware, but that's more options, unnecessary too, just compose tools.
Principle of least surprise and all that.
I wasn't expecting the downvotes for an xkcd reference
It’s really the dream UI/UX from sience fiction movies: “take all images from this folder and crop 100px away except on top, saturate a bit and save them as uncompressed tiffs in this new folder, also assemble them in a video loop, encode for web”.
A prompt to ChatGPT and a command later and all were nicely cropped in a second.
The dread of doing it by hand and having it magically there a minute later is absolutely mind blowing. Even just 5 years ago, I would have just done it manually as it would have definitely taken more to write the code for this task.
If you don't care enough about potential side effects to read the manual it's fine, but a dream UX it is not because I'd argue that includes correctness.
https://youtu.be/9kaIXkImCAM?si=b_vzB4o87ArcYNfq
Handbrake and Losslssscut are great too. But in addition to donating to FFmpeg, I pay for ffWorks because it really does offer a lot of value to me. I don’t think there is anything close to its polish on other platforms, unfortunately.
[1]: https://www.ffworks.net/index.html
It's a great tool. Little long in the tooth these days, but gets the job done.
Past that, I'm on the command line haha
[0] https://handbrake.fr
https://www.shotcut.org/
https://www.mltframework.org/
Could be an interesting data source to explore that opinion.
Then it stopped working until I updated youtube-dl and then that stopped working once I lost the incantation :<
[0] - https://xkcd.com/2347/
[0] https://link.springer.com/article/10.1007/s11214-020-00765-9
That being said, if you put down a pie chart of media frameworks (especially for transcoding or muxing), ffmpeg would have a significant share of that pie.
Linux doesn't really have a system codec API though so any Linux video software you see (ex. VLC, Handbrake) is almost certainly using ffmpeg under the hood (or its foundation, libavcodec).
It also was originally authored by the same person who did lzexe, tcc, qemu, and the current leader for the large text compression benchmark.
Oh, and for most of the 2010's there was a fork due to interpersonal issues on the team.
It’s exceedingly good software though, and to be fair I think it’s gotten a fair bit of sponsorship and corporate support.
This seemed to be interesting to users of this site. tl;dr they added support for whisper, an OpenAI model for speech-to-text, which should allow autogeneration of captions via ffmpeg
yep, finally the deaf will able to read what people are saying in a porno!
This could streamline things
I am curious about adoption and features that would make big difference to users :)
Secondly, just curious: any insiders here?
What changed? I see the infrastructure has been upgraded, this seems like a big release, etc. I guess there was a recent influx of contributors? A corporate donation? Something else?
[1]: https://github.com/ggml-org/whisper.cpp
[2]: https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/13ce36fef98a...
https://news.ycombinator.com/item?id=44886647 ("FFmpeg 8.0 adds Whisper support (ffmpeg.org)"—9 days ago, 331 comments)