Show HN: I’ve built an IoT device to let my family know when I’m in a meeting (nullonerror.org)

54 points by delduca 2d ago 36 comments

Show HN: HelixDB – Open-source vector-graph database for AI applications (Rust) (github.com)

140 points by GeorgeCurtis 11h ago 63 comments

Show HN: Mycelium (github.com)

3 points by brendoncarroll 3d ago 0 comments

Show HN: A5 (github.com)

83 points by pheelicks 19h ago 26 comments

Show HN: Airweave – Let agents search any app (github.com)

158 points by lennertjansen 1d ago 37 comments

Show HN: Basecoat – shadcn/UI components, no React required

113 points by hunvreus 17h ago 65 comments

Show HN: AG-UI Protocol – Bring Agents into Frontend Applications (github.com)

23 points by swiftlyTyped 13h ago 5 comments

Show HN: Lumoar – Free SOC 2 tool for SaaS startups (lumoar.com)

87 points by asdxrfx 1d ago 31 comments

Show HN: CLI that spots fake GitHub stars, risky dependencies and licence traps (github.com)

117 points by artski 1d ago 68 comments

Show HN: I made a site for finding people to build cool tech projects with (guildorigin.com)

14 points by quelup 14h ago 14 comments

Show HN: Litelytics – A nice GA4 dashboard (better Looker Studio) (litelytics.io)

6 points by tesfaX 9h ago 1 comments

Show HN: Put macros.menu/ in front of any restaurant menu URL (macros.menu)

6 points by Sawpaw19 9h ago 0 comments

Show HN: Make your own voice AI in two clicks (unmute.sh)

20 points by unmute-sh 17h ago 5 comments

Show HN: Codigo – The Programming Language Repository (codigolangs.com)

78 points by adamjhf 3d ago 24 comments

Show HN: Downloads Director – Auto-organize your Downloads folder by file type

4 points by freakynit 14h ago 1 comments

Show HN: GlassFlow – OSS streaming dedup and joins from Kafka to ClickHouse (github.com)

76 points by super_ar 2d ago 31 comments

Show HN: The missing inbox for GitHub pull requests (github.com)

25 points by pvcnt 1d ago 5 comments

Show HN: Rama 0.2 – Modular Rust framework for proxies, servers, and clients (github.com)

2 points by gdcbe 16h ago 0 comments

Show HN: LoopMix128 – Fast C PRNG (.46ns), 2^128 Period, BigCrush/PractRand Pass (github.com)

75 points by the_othernet 3d ago 34 comments

Show HN: Blog comments, nice looking, open source – Talkyard (blog-comments.talkyard.io)

22 points by KajMagnus 1d ago 3 comments

Show HN: Translate between dialects and share SQL queries on the browser (sqlscope.netlify.app)

17 points by WhyIsItAlwaysHN 5d ago 2 comments

Show HN: Cyckle (v1.2.0): CSV analysis and more (github.com)

2 points by vd2287 8h ago 1 comments

Show HN: MMORPG prototype inspired by World of Warcraft (github.com)

25 points by nickyvanurk 1d ago 9 comments

Show HN: Code Claude Code (github.com)

117 points by sean_ 3d ago 25 comments

Show HN: Hydra (YC W22) – Serverless Analytics on Postgres (hydra.so)

59 points by coatue 4d ago 33 comments

Show HN: Oliphaunt – A native Mastodon client for macOS (testflight.apple.com)

103 points by anosidium 4d ago 38 comments

Show HN: An API for LLM capabilities and pricing, scraped from docs (paolino.me)

2 points by earcar 20h ago 0 comments

Show HN: Xenolab – Rasp Pi monitor for my pet carnivourus plants (github.com)

131 points by malux85 3d ago 53 comments

Show HN: Hyvector – A fast and modern SVG editor (hyvector.com)

326 points by jansan 4d ago 91 comments

Show HN: A backend agnostic Ruby framework for building reactive desktop apps (codeberg.org)

92 points by zero-st4rs 4d ago 34 comments

Show HN: BlenderQ – A TUI for managing multiple Blender renders (github.com)

62 points by TechSquidTV 4d ago 4 comments

Show HN: I’m 16 years old and working on my first startup, a study app (notiv.app)

81 points by WilliamCranna 2d ago 125 comments

Show HN: Hyper – Standards-first React alternative (nuejs.org)

70 points by tipiirai 4d ago 84 comments

Show HN: Triplex, a visual workspace for React / Three Fiber (github.com)

3 points by madou 1d ago 0 comments

Show HN: Using eBPF to see through encryption without a proxy (github.com)

258 points by tylerflint 5d ago 78 comments

Show HN: Open-source MMORPG prototype inspired by World of Warcraft (everwilds.io)

6 points by nickyvanurk 1d ago 2 comments

Show HN: Vom Decision Platform (Cursor for Decision Analyst) (vomdecision.com)

15 points by davidreisbr 5d ago 13 comments

Show HN: Aberdeen – An elegant approach to reactive UIs (aberdeenjs.org)

223 points by vanviegen 4d ago 128 comments

Show HN: US Routing – Python library for fast local routing in the US (github.com)

114 points by ivanbelenky 6d ago 26 comments

Show HN: LLM-God – An LLM Multiprompting App (github.com)

4 points by czhou578 1d ago 0 comments

Show HN: AGI Hits a Structural Wall – A Billion-Dollar Problem

8 points by mmschlereth 1d ago 0 comments

Show HN: Sprigman – Pac-Man Recreated in a Limited Tile Based JavaScript Engine (sprig.hackclub.com)

19 points by kuberwastaken 6d ago 6 comments

Show HN: Open-source AI code review agent that's aware of your entire codebase (docs.sourcebot.dev)

5 points by msukkarieh 1d ago 1 comments

Show HN: I Built Remind Me AI. It's Like Unlimited GPT Tasks. Try the Demo (app.arcade.software)

5 points by ShawnBuilds 2d ago 0 comments

Show HN: GS-Base – A multifunctional database tool with Python integration (citadel5.com)

2 points by jpiech 1d ago 0 comments

Show HN: Vibemail – Chrome extension for vibe emailing (chromewebstore.google.com)

9 points by aliyanishfaq 1d ago 7 comments

Show HN: Jester – An RSS/Atom Reader and Content Management Tool (jesterengine.com)

4 points by dan_h 1d ago 0 comments

Show HN: TheorIA – An Open Curated Physics Dataset (Equations,Explanations,JSON) (theoria-dataset.github.io)

9 points by ManuelSH 2d ago 6 comments

Show HN: Eurker – A Developer-First Uptime Monitoring Platform with JSON Configs (eurker.com)

3 points by ajoshu 1d ago 0 comments

Show HN: Downloads and saves any YouTube videos and playlists (github.com)

4 points by pierres7 1d ago 0 comments

Ask HN: Web scraping in production?

3 arkmm 5 5/13/2025, 8:29:26 PM

Are any of you maintaining any web scrapers in production?

I've done some for side projects, automated testing, and personal scripts (checking personal bank balances, getting a Global Entry interview slot, etc.), but it always feels very brittle.

Curious what applications people have in industry and what sorts of techniques people use for reliability.

Comments (5)

9d · 8h ago

Scraping is inherently brittle, but it can be very useful for short-term scraping in very specific circumstances. I haven't had any in maybe 10 years.

sargstuff · 8h ago

IMHO, "untyped" format/delimited file yes. Directly placing/'compiling' in appropriate topological construct/environment works wonders. aka environment of database, spreadsheet, "reports" with information beyond raw data, etc

sargstuff · 8h ago

excel web scraping[0] (vs. using python[1] and/or odbc/delimited files)

A few 2025 use cases [2],[3]:

   Use publically available database information (construction, taxes, sales, traffic report, proposed building/zone changes, etc) to find out what's going on withing an area aka. zip code, housing area, 'vacation spot', etc

----

   creative take on topic:

      modern looming / static 'threaded' approach : https://news.ycombinator.com/item?id=43977384

      Structurally reprogrammable magnetic maetamaterials hold promise for biomedicine, soft robotics. ("web" support formed via scraping material in relevant patterns) : https://techxplore.com/news/2025-05-reprogrammable-magnetic-metamaterials-biomedicine-soft.html

      3d printed smart-fabrics : https://techxplore.com/news/2025-05-d-smart-fabrics-flexibility-ability.html

----

[0] : excel scraping : https://www.youtube.com/watch?app=desktop&v=6coVzIt93vk

[1] : python scraping : https://www.youtube.com/watch?v=Oo8-nEuDBkk

[2] : https://dataforest.ai/blog/top-web-scraping-use-cases

[3] : https://www.parsehub.com/blog/web-scraping-examples/

arkmm · 8h ago

Neat - didn't realize there were affordances for scraping in Excel (but in hindsight I shouldn't be surprised).

I didn't follow the connection between modern looming and scraping though?

sargstuff · 8h ago

hint: silk spider webs & fabric threads.

Guess 3d printing should have been clarified as linear, fused deposition. Melted plastic line gets scraped along plate/material.

The 3d printed web reference, in this instance, being the in-fill pattern. [0]

Robotic metal pinching / incremental sheet forming might be bit more clearer example. [1]

-----

[0] : in-fill pattern : https://jlc3dp.com/blog/choosing-the-right-infill-structure-...

[1] : https://www.youtube.com/watch?v=Jc16Ob-yoDs