I built an app that uses AI to fact check Podcasts and YouTube videos while you watch.
I got sick of listening to podcasts and not knowing if their hosts or guests were full of BS.
My hope is that unbiased, dispassionate AI fact checking agents can help us sift through all the BS and get to the truth, holding the podcast community to a higher standard in the process.
I'd love your help in making this better!
latexr · 7h ago
Accuracy, truth, and trust are paramount for fact checking, and LLMs are notoriously unreliable at all of them.
This is going to have the opposite effect of your stated goal. It will only serve to amplify confusion and have more people confidently defend “facts” which are complete bullshit.
It was immediately obvious something was amiss when on your own homepage the videos with “0% incorrect claims” are from Trump and Tucker Carlson, and then the John Oliver video on AI slop is somehow “19% incorrect claims”. So I clicked on it, and your page showed me the supposed false claims, and its sources were all 404 broken links.
tycoop81 · 7h ago
I appreciate the feedback. LLM hallucination is definitely a concern. But, my hope is that on balance, their ability to rapidly sift through a litany of claims and check those against high integrity journalistic sources (peer reviewed journals, etc), will help people sift through a world increasingly filled with BS.
Are these the Trump and Tucker videos you are referencing?
These are the only ones on the home page with 0% incorrect claims. While Tucker and Trump are featured in the thumbnails, the videos are actually from NPR and The Young Turks. On first glance, I don't see any missed "BS" in either video's reporting, but please let me know if I missed something as I want to make this system better every day.
Thanks for your feedback!
tycoop81 · 7h ago
I'm looking at the John Oliver video now and see a host of 404s in the info card sources, as you did. I'm going to work on the prompt / system to solve for this. Great catch!
I got sick of listening to podcasts and not knowing if their hosts or guests were full of BS.
My hope is that unbiased, dispassionate AI fact checking agents can help us sift through all the BS and get to the truth, holding the podcast community to a higher standard in the process.
I'd love your help in making this better!
This is going to have the opposite effect of your stated goal. It will only serve to amplify confusion and have more people confidently defend “facts” which are complete bullshit.
It was immediately obvious something was amiss when on your own homepage the videos with “0% incorrect claims” are from Trump and Tucker Carlson, and then the John Oliver video on AI slop is somehow “19% incorrect claims”. So I clicked on it, and your page showed me the supposed false claims, and its sources were all 404 broken links.
Are these the Trump and Tucker videos you are referencing?
https://bsmtr.com/video/zdMBnGxSsLw
https://bsmtr.com/video/RH6Po1jiwMU
These are the only ones on the home page with 0% incorrect claims. While Tucker and Trump are featured in the thumbnails, the videos are actually from NPR and The Young Turks. On first glance, I don't see any missed "BS" in either video's reporting, but please let me know if I missed something as I want to make this system better every day.
Thanks for your feedback!