Ask HN: Can LLMs do batch classification?
1 iknownthing 1 6/26/2025, 1:16:29 AM
I wrote a prompt that did batch classification - the prompt contained instructions on how to classify text and 10 input examples for it to classify and it was to return a json string displaying the classifications. It kind of worked by then I realized the individual classification of an individual input was significantly affected by which other 9 inputs it was in the prompt with. In other works the classification was not at all independent. With traditional ML you can do batch classification trivially with each input in the batch being predicted independently. So is this just a limitation of LLMs? You have to classify inputs one LLM call at a time?
It sounds like we may have hit similar limits, using slightly different means to get there.