This is a super cool project. But it would be 10x cooler if they had generated CLIP or some other embeddings for the images, so you could search for text but also do semantic vector search like "people fighting", "cats and dogs, "red tesla", "clown", "child playing with dog", etc.
jacobajit · 21m ago
I feel like street-view data is surprisingly underused for geospatial intelligence.
With current-gen multimodal LLMs, you could very easily query and plot things like "broken windows," "houses with front-yard fences," "double-parked cars," "faded lane markers," etc. that are difficult to generally derive from other sources.
For any reasonably-sized area, I'd guess the largest bottleneck is actually the Maps API cost vs the LLM inference. And ideally we'd have better GIS products for doing this sort of analysis smoothly.
m_kos · 3h ago
GitHub of the person who prepared the data. I am curious how much compute was needed for NY. I would love to do it for my metro but I suspect it is way beyond my budget.
(The commenters below are right. It is the Maps API, not compute, that I should worry about. Using the free tier, it would have taken the author years to download all tiles. I wish I had their budget!)
LeifCarrotson · 2h ago
I would wager the compute for the OCR is cheap. Just get a beefy local desktop PC, if it runs overnight or even takes a week that's fine.
It's the Google Maps API costs that will sink your project if you can't get them waived as art:
Not sure how many panoramas there are in New York or your metro, but if it's over the free tier you're talking thousands of dollars.
daemonologist · 2h ago
The linked article mentions that they ingested 8 million panos - even if they're scraping the dynamic viewer that's $30k just in street view API fees (the static image API would probably be at least double that due to the low per-call resolution).
OCR I'd expect to be comparatively cheap, if you weren't in a hurry - a consumer GPU running PaddlePaddle server can do about 4 MP per second. If you spent a few grand on hardware that might work out to 3-6 months of processing, depending on the resolution per pano and size of your model.
ks2048 · 2h ago
It says 8 million images. So, 13.2 images/second for one week.
I'm wondering about more the data - did they use Google's API or work with Google to use the data?
A game: find an English word with the fewest hits. (It must have at least one hit that is not an OCR error, but such errors do still count towards your score. Only spend a couple of minutes.) My best is "scintillating" : 3.
cmwelsh · 37m ago
First lucky try, “calisthenics” scores a verified 1. It would be interesting if there was a Parquet file of the raw data.
This is pretty cool!
I'm curious what was used for OCR? Amazon Mechanical Burp?
tills13 · 3h ago
I _love_ this but it's pretty bad. I searched for "Morgue" and one of the matches was the "2025 Google" watermark which it thought was "Big Morgue"
Again, a complex problem and I love it...
cobbzilla · 2h ago
Searching for “foo” is humorous, it’s mostly restaurants with signs that say “food” but the “d” is cropped.
zxh · 15m ago
When you search 'google'... you'll see... lol
shibeprime · 2h ago
520 matches on "hotdog"
8084 matches on "massage"
in no particular order
IAmGraydon · 3h ago
As others have mentioned, the idea is so cool, but the text recognition is abysmal.
lelandfe · 1h ago
It worked perfectly on the two tests I tried: the GSA building in SoHo, and BKLYN Blend in Bedstuy.
ya1sec · 2h ago
amazing. look up some graffiti writers you know
8bitsrule · 1h ago
Gosh! Maybe one of these days someone will take time off from this cultural wonderment to construct a simple, easy to use, text-to-audio.file program - you know, install, paste in some text, convert, start-up a player - so that the blind can listen to texts that aren't recorded in audiobooks. Without a CS degree.
repeekad · 41m ago
I think the issue is the compute power needed for good voice models is far from free just in hardware and electricity, so any good text to audio solution likely needs to cost some money. Wiring up Google vertex AI text to speech or the aws equivalent is probably something chat gpt could walk most people through even without a CS degree, a simple python script you could authenticate from a terminal command, and would maybe cost a couple bucks for personal usage
A service you can pay for of that simplicity probably doesn’t exist because there are other tools that integrate better with how the blind interact with computers, I doubt it’s copy and pasting text, and those tools are likely more robust albeit expensive
egypturnash · 2h ago
I typed in "fart" and none of the results on the first page were actually the word "fart".
https://www.alltext.nyc/search?q=Sex
With current-gen multimodal LLMs, you could very easily query and plot things like "broken windows," "houses with front-yard fences," "double-parked cars," "faded lane markers," etc. that are difficult to generally derive from other sources.
For any reasonably-sized area, I'd guess the largest bottleneck is actually the Maps API cost vs the LLM inference. And ideally we'd have better GIS products for doing this sort of analysis smoothly.
https://github.com/yz3440
(The commenters below are right. It is the Maps API, not compute, that I should worry about. Using the free tier, it would have taken the author years to download all tiles. I wish I had their budget!)
It's the Google Maps API costs that will sink your project if you can't get them waived as art:
https://mapsplatform.google.com/pricing/
Not sure how many panoramas there are in New York or your metro, but if it's over the free tier you're talking thousands of dollars.
OCR I'd expect to be comparatively cheap, if you weren't in a hurry - a consumer GPU running PaddlePaddle server can do about 4 MP per second. If you spent a few grand on hardware that might work out to 3-6 months of processing, depending on the resolution per pano and size of your model.
I'm wondering about more the data - did they use Google's API or work with Google to use the data?
IIRC he found a way to download streetview images without paying, and used the OCR built-in to macOS (which is really good).
All Text in NYC - https://news.ycombinator.com/item?id=42367029 - Dec 2024 (4 comments)
All text in Brooklyn - https://news.ycombinator.com/item?id=41344245 - Aug 2024 (50 comments)
A game: find an English word with the fewest hits. (It must have at least one hit that is not an OCR error, but such errors do still count towards your score. Only spend a couple of minutes.) My best is "scintillating" : 3.
https://www.alltext.nyc/search?q=Calisthenics
https://www.alltext.nyc/search?q=perplexed
Instead shows me thousands of “Rev“
Again, a complex problem and I love it...
A service you can pay for of that simplicity probably doesn’t exist because there are other tools that integrate better with how the blind interact with computers, I doubt it’s copy and pasting text, and those tools are likely more robust albeit expensive