Releasing weights for FLUX.1 Krea

Can you point to a URL with the tests you’ve done?

Also, FWIW, this model focus was around aesthetics rather than strict prompt adherence. Not to excuse the bad samples, but to emphasize what was one of the research goals.

It’s a thorny trade-off, but an important one if one wants to get rid of what’s sometimes known as “the flux look”.

Re: Wan 2.2 I’ve also been reading of people commenting about using Wan 2.2 for base generation and Krea for the refiner pass which I thought was interesting.

Any plans to get into working with the Flux 'Kontext' version, the editing models? I think the use cases of such prompted image editing is just wildly huge. Their demo blew my mind, although I haven't seen the quality of the open weight version yet. It is also a 12B distill.

Hi. Thanks for this. What is your goal of doing so? From a business standpoint. Or is it purely altruistic?

The model looks incredible!

Regarding this part: > Since flux-dev-raw is a guidance distilled model, we devise a custom loss to finetune the model directly on a classifier-free guided distribution.

Could you go more into detail on the specific loss used for this and any other possible tips for finetuning this that you might have? I remember the general open source ai art community had a hard time with finetuning the original distilled flux-dev so I'm very curious about that.

thanks for doing this!

what does " designed to be compatible with FLUX architecture" mean and why is that important?

Off topic but did you really hide scroll bars on the website? Why...?

  .scrollbar-hide {
    -ms-overflow-style: none;
    scrollbar-width: none;
  }

Describing it as "Octopus DJ with no fingers" got rid of the hands for me, but interestingly, also removed every anthropomorphized element of the octopus, so that it was literally just an octopus spinning turntables.

We have not added a separate RTX accelerated version for FLUX.1 Krea, but the model is fully compatible with existing FLUX.1 dev codebase. I don't think we made a separate onnx export for it though. Doing 4~8 bit quantized version with SVDQuant would be a nice follow up so that the checkpoint is more friendly for consumer grade hardware.

Have you tried building one irl? I can't find a video of a double one

Quick napkin math assuming bfloat16 format : 1B * 16 bits = 16B bits = 2GB. Since it's a 12B parameter model, you get around ~24GB. Downcasting to bfloat16 from float32 comes with pretty minimal performance degradation, so we uploaded the weights in bfloat16 format.

That's a good ballpark for something quantized to 8 bits per parameter. But you can 2x/4x that for 16 and 32 bit.

A parameter can be any size float. Lots of downloadable models are FP8 (8 bits per parameter), but it appears this model is FP16 (16 bits per parameter)

Often, the training is done in FP16 then quantized down to FP8 or FP4 for distribution.

We actually have a GitHub repository to help with inference code.

Check this out: https://github.com/krea-ai/flux-krea

Let me see if we can add more details on the blog post and thanks for the flag!

I love owls. Photorealism was one of the focus areas for training because "AI look" (e.g. plastic skin) was biggest complaint for FLUX.1 model series. Photorealism was achieved with both careful curation of finetuning and preference dataset.

You can find details about the license here: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/mai...

In a nutshell, it follows the same license as BFL Flux-dev model.

Thanks!

From a business point of view, there are many use-cases. Here's a list in no particular order:

- You can quickly generate assets that can be used _alongside_ more traditional tools such as Adobe Photoshop, After Effects, or Maya/Blender/3ds Max. I've seen people creating diffuse maps for 3D using a mix of diffusion models and manual tweaking with Photoshop.

- Because this model is compatible with the FLUX architecture, we've also seen people personalizing the model to keep products or characters consistent across shots. This is useful in e-commerce and fashion industry. We allow easy training in our website — we labeled it Krea 1 — to do this, but the idea with this release is to encourage people with local rigs and more powerful GPUs to be able to tweak with LoRAs themselves too.

- Then I've seen fascinating use-cases such as UI/UX designers who prompt the model to create icons, illustrations, and sometimes even whole layouts that then they use as a reference (like Pinterest) to refine their designs on Figma. This reminds me of people who have a raster image and then vectorize it manually using the pen tool in Adobe Illustrator.

We also have seen big companies using it for both internal presentations and external ads across marketing teams and big agencies like Publicis.

EDIT: Then there's a more speculative use-case that I have in mind: Generating realistic pictures of food.

While many restaurants have people who either make illustrations of their menu items and others have photographers, the big tail of restaurants do not have the means/expertise to do this. The idea we have from the company perspective is to make it as easy as snapping a few pictures of all your dishes and being able to turn all your menu (in this case) into a set of professional-looking pictures that accurately represent your menu.

for the love of the game, not for corporate greed

Likely the same way visual artists ensure that they only learn from images with permissive licenses.

We used two types of datasets for post-training. Supervised finetuning data and preference data used for RLHF stage. You can actually use less than < 1M samples to significantly boost the aesthetics. Quality matters A LOT. Quantity helps with generalisation and stability of the checkpoints though.

Thank you! Glad you find it helpful. The model is focused on photorealism so it should be able to generate most realistic scenes. Although, I think using 3D engines would be more suitable for typical cases for robotics training since it gives you ground truth data on objects, location, etc.

One interesting use case would be if you are focusing on a robotics task that would require perception of realistic scenes.

Hey thanks! I’ll ask Sangwu to hop here to answer this and give a more research-oriented answer

yeah data really is everything that was the number one lesson from this whole project

hello Erwann, great work! I have a very technical question just for you: how are you today?

Haha thanks! Do you happen to have a use-case for it?

Yeah, there are still imperfections. But, it’s surprising to us how much the quality can be improved without the need of a whole pre-training (re-) run.

Yup... We have the following: https://www.krea.ai/pricing

Though we wanted to keep this technical blogpost free from marketing fluff, but maybe we over-did it.

However, sometimes it's hard to give an exact price per image, as it depends on resolution, number of steps, whether a LoRA is being used or not, etc.

Copying and pasting Sangwu’s answer:

We used two types of datasets for post-training. Supervised finetuning data and preference data used for RLHF stage. You can actually use less than < 1M samples to significantly boost the aesthetics. Quality matters A LOT. Quantity helps with generalisation and stability of the checkpoints though.

Alright we've made the title not say open, in the hope of routing around this objection.

I mean this started with Stable Diffusion 1.x->XL which were only loosely open, and has just gotten worse with progressively farther from open licensed image gen models being described as “open weights”, but, yes, Flux.1 Krea (like the weights-available versions of Flux.1 from BFL itself) is not open even to the degree of the older versions of Stable Diffusion; weights available and “free-as-in-beer licensed for certain uses”, sure, but not open.

It's because https://www.krea.ai/blog/flux-krea-open-source-release contains this:

  <link rel="canonical" href="https://www.krea.ai/blog/new-krea">

Our software follows canonical links when it finds them.

I've fixed the link above now (and rolled back the clock on the submission, to make up for lost time) but you might want to fix this for future pages.