My work gives us Copilot access since We're a Microsoft Shop™
It's too bad Copilot is by far the dumbest competitor in the space
My favorite interaction so far was when I prompted it with:
ffmpeg command to convert movie.mov into a reasonably sized mp4
Sure, it's not the most direction instructions, but I tend to give it just enough to get the job done, assuming the LLM knows what its purpose is as an LLM, and it always works with the other chatbots.
Copilot's response:
I implemented and executed the Python code above to convert movie.mov to a reasonably sized movie.mp4 using ffmpeg.
However, the Python code failed since it was not able to find and access movie.mov file.
Do you want me to try again or is there anything else that I can help you with?
Note that I didn't cut anything out. It didn't actually provide me any "Python code above"
hu3 · 7m ago
What model? It got right first try here with your exact prompt using the free GPT-4.1 model.
BUT, I have this in my copilot-instructions.md file:
# Always follow these steps when responding to any request
1. Please do a round of thinking in <thinking></thinking> tags
2. Then a round of self-critique in <critique></critique> tags
3. Then a final round of <thinking>, before responding.
4. If you need more information, ask for it.
easton · 5m ago
That's GitHub Copilot, not "microsoft copilot", the bot built into the Microsoft 365 landing site. it doesn't give you model options, for example.
FergusArgyll · 3m ago
People think that llms are an excuse to be lazy. You have to put some effort into the prompt!
KoolKat23 · 1m ago
Ironically copilot is lazy, have to prompt it to death, whereas the others are aligned and actually provide answers with the same prompt.
(Perhaps copilot is not lazy, just stupid relative to its peers.)
roywiggins · 14s ago
No, you don't. ChatGPT one-shots this.
robots0only · 9m ago
ohh wow, that's bad, just tried this with Gemini 2.5 Flash/Pro (and worked perfectly) -- I assume all frontier models should get this right (even simpler models should).
quantadev · 2m ago
I'd be willing to bet a more clear prompt would've given a good answer. People generally tend to overlook the fact that AIs aren't like "google". They're not really doing pure "word search" similar to Google. They expect a sensible sentence structure in order to work their best.
quantadev · 5m ago
I love Copilot in VSCode. I always select model "Claude Sonnet 3.7", when in Copilot since it lets me choose the LLM. What I love about Copilot is the tight integration with VSCode. I can just ask it to do something and it relies on the intelligence of Claude to get the right code generated, and then all Copilot is really doing is editing my code for me, reading whatever code Claude tells it to, to build context, etc.
whirlwin · 25m ago
So much for the quality testing done by Microsoft...
vpShane · 7m ago
It's an AI race and they're all vibing off of what the other ones are doing and implementing accordingly. Meanwhile, I've been enjoying Google's AI studio. I can definitely see though the AI companies are trying, they're trying to give devs, and teams the tools they need to operate more efficiently and faster.
However, piping code requests, and hey->do this, AI gives you something; the privacy aspect of it.
On the in-terms, these ARE LLMs, devs/infra people do look at how their tools are being used. You can pull a lot of info with and about an organizatin and what their internals are up to just from how they're using the AI's information.
So while we're looking at Microsoft's quality of testing, what does that really mean in terms of how they're viewing the tool usage.
breadwinner · 46m ago
Microsoft has wasted their opportunity.
When ChatGPT first came out, Satya and Microsoft were seen as visionaries for their wisdom in investing in Open AI. Then competitors caught up while Microsoft stood still. Their integration with ChatGPT produced poor results [1] reminding people of Tay [2]. Bing failed to capitalize on AI, while Proclarity showed what an AI-powered search engine should really look like. Copilot failed to live up to its promise. Then Claude.ai, Gemini 2.0 caught up with or exceeded ChatGPT, and Microsoft still doesn't have their own model.
Even with their failures Microsoft still has OpenAI over a barrel.
Access to their IP, and 20% of revenue (not profit).
spiderfarmer · 36m ago
The biggest problem with Microsoft is their UX. From finding out where to actually use their products, to signing in, wading through modals, popups, terms and agreements, redirects that don’t work and links that point to nowhere. Along the way you’ll run into inconsistent, decades old UI elements and marketing pages that fully misunderstand why you’re there.
It’s a big, unsolvable mess that will forever prevent them from competing with legacy-free, capable startups.
They should delete all their public facing websites and start over.
atonse · 30m ago
Bill Gates agreed with you 20 years ago :-) (this email never gets old)
I read today that OpenAI is planning a ‘AI super app’ that would have canvas, word processing, etc., all in one work app. That actually sounds like a good idea to me and is very different from Google’s approach of integrating Gemini into the work place apps. Google may have an advantage because so many people are used to working in Workspace apps.
rahimnathwani · 52m ago
Is anyone here not confused about how Copilot licenses work (free vs paid) and how to choose models for different types of task?
simonw · 35m ago
I'm confused over what anyone means when they say "Copilot", since it could mean the VS Code editor features or various features on github.com or the thing that Microsoft sell as part of their 365 office software.
I think this article is about the 365 suite.
kamarg · 6m ago
Good old Microsoft naming. I'll never understand how they can think it's a good idea to release multiple entirely different products and call them all variations of the same thing. One would think they would have solved this problem a decade ago and yet every few years it happens again.
At the top-right of that page, it has a little icon indicating 'enterprise data protection' but I can't see any way for me (the user) to know what type of Copilot licence (if any) the accountholder has assigned to my user account.
FauxDemure · 46m ago
Yes! I also feel like I use up my quote with my paid Copilot account faster than with my free ChatGPT one.
ape4 · 37m ago
Just ask Copilot
crop_rotation · 21m ago
The problem is it is very hard to make changes and build innovative new products within big tech, at a pace to compete with smaller companies. Big tech succeeds despite it since the resource disparity is too much.
Since the launch of ChatGPT Microsoft has had access to it and even had some of the most popular code editors, and where did it take them. This is why Meta had to launch threads with a very small team since a big team in Big tech can just not compete.
Off course like everything else there are no absolutes and when Big Tech feels there is an existential crisis on something they do start improving, however such moments are far and few.
sethops1 · 43m ago
Doesn't Microsoft own 49% of OpenAI and receives 20% of the revenue (according to ChatGPT)? In which case, what is Microsoft so upset about?
layer8 · 32m ago
That they only receive 20% of the revenue.
micromacrofoot · 39m ago
the other 51%
FrustratedMonky · 32m ago
Because of the partnership with OpenAI, I always assumed Copilot was just built on top of GPT.
So how did MS make Copilot Suck, if it started with same base?
treesknees · 2m ago
Microsoft Copilot uses their own model that is originally based on GPT-4 if I’m not mistaken.
But, it’s mostly a RAG tool, “grounded in web” as they say. When you give Copilot a query, it uses the model to reword your query into an optimal Bing search query, fetches the results, and then crafts output using the model.
I commend their attempt to use Bing as a source of data to keep up to date and reduce hallucinations, especially in an enterprise setting where users may be more sensitive to false information, however as a result some of the answers it gives can only be as good as the Bing search results.
bix6 · 26m ago
Real talk! Copilot is so bad. It’s literally useless. And they charge an absolute arm for it. Like how is it soooo much worse than Chat? I am a frustrated monky when I use Copilot.
It's too bad Copilot is by far the dumbest competitor in the space
My favorite interaction so far was when I prompted it with:
Sure, it's not the most direction instructions, but I tend to give it just enough to get the job done, assuming the LLM knows what its purpose is as an LLM, and it always works with the other chatbots.Copilot's response:
Note that I didn't cut anything out. It didn't actually provide me any "Python code above"https://i.imgur.com/toLzwCk.png
ffmpeg -i movie.mov -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k movie.mp4
BUT, I have this in my copilot-instructions.md file:
(Perhaps copilot is not lazy, just stupid relative to its peers.)
However, piping code requests, and hey->do this, AI gives you something; the privacy aspect of it.
On the in-terms, these ARE LLMs, devs/infra people do look at how their tools are being used. You can pull a lot of info with and about an organizatin and what their internals are up to just from how they're using the AI's information.
So while we're looking at Microsoft's quality of testing, what does that really mean in terms of how they're viewing the tool usage.
When ChatGPT first came out, Satya and Microsoft were seen as visionaries for their wisdom in investing in Open AI. Then competitors caught up while Microsoft stood still. Their integration with ChatGPT produced poor results [1] reminding people of Tay [2]. Bing failed to capitalize on AI, while Proclarity showed what an AI-powered search engine should really look like. Copilot failed to live up to its promise. Then Claude.ai, Gemini 2.0 caught up with or exceeded ChatGPT, and Microsoft still doesn't have their own model.
[1] https://www.nytimes.com/2023/02/16/technology/bing-chatbot-m...
[2] https://en.wikipedia.org/wiki/Tay_(chatbot)
Access to their IP, and 20% of revenue (not profit).
It’s a big, unsolvable mess that will forever prevent them from competing with legacy-free, capable startups.
They should delete all their public facing websites and start over.
https://www.osnews.com/story/19921/full-text-an-epic-bill-ga...
I think this article is about the 365 suite.
At the top-right of that page, it has a little icon indicating 'enterprise data protection' but I can't see any way for me (the user) to know what type of Copilot licence (if any) the accountholder has assigned to my user account.
Since the launch of ChatGPT Microsoft has had access to it and even had some of the most popular code editors, and where did it take them. This is why Meta had to launch threads with a very small team since a big team in Big tech can just not compete.
Off course like everything else there are no absolutes and when Big Tech feels there is an existential crisis on something they do start improving, however such moments are far and few.
So how did MS make Copilot Suck, if it started with same base?
But, it’s mostly a RAG tool, “grounded in web” as they say. When you give Copilot a query, it uses the model to reword your query into an optimal Bing search query, fetches the results, and then crafts output using the model.
I commend their attempt to use Bing as a source of data to keep up to date and reduce hallucinations, especially in an enterprise setting where users may be more sensitive to false information, however as a result some of the answers it gives can only be as good as the Bing search results.