Ask HN: How to visualize all human thought
1. Tag all content with a date (probably almost done). In long passages, estimate a date based on published + written rate. 2. Once every sentence or phrase has a date, create a few blockchains. One for short thoughts (128 tokens), and more chains for longer thoughts (3072+ tokens). 3. Run an LLM embeddings cosine similarity or some better metric for each tokenized idea with a threshold of say 75%, tokenized by natural punctuation if not found. 4. Only NEWish content gets stored in the block, chained if novel. Again, short and long thoughts recorded. Coordinates from the vectors. 5a. I remember reading once that working through a PhD was like pricking the inside of a balloon, I love that visualization. Every novel idea pricks the inside of a growing sphere of knowledge over time. 5b. Has anyone ever tried to map human knowledge in three dimensions? 5c. Alternatively, a tree or root system growing over time. Branching from the vectors.
1) human knowledge is “holographic” not like a block chain. There are no dates, and the ability to compare and relate is much different than block chain serialization.
2) much of what we know is WRONG! Even if only a little. Things like “big bang” and “states of mind” and “dark matter” and so much more are cloudy imaginations that blur into so called science, yet you would include them as “knowledge” when they are fantasy suturing our ignorance and confusion.
3) you have more than three dimensions to work with. Any vector attribute is a dimensionality when graphing so color, size, line thicknesses, solidity, etc may be useful considerations while “graphing”. The mind is more like this, a hyperdimensional holographic interrelationships.
4) in your idea you would have one representation, in the human mind there are many representations even those contradicting each other. Think about those who both believe in God and Science, or humans and peaceful coexistence (we are natural competitors who covet what we do not have), or even reconcile how words and deeds do not align.
Much of our “knowledge” is artificial and prescribed, still more is usefulness not necessarily accuracy, still more relative to those who accept whatever it is as a principle of something else unrelated.
Not even science can cut through the ambiguity and contradiction that is human thought or “knowledge”.
Have fun with it though!
All this stuff about tokens and vectors is quite different from my approach. I'm always interested to see how other metamages approach this issue.
I think a hypergraph structure is the only thing capable of properly representing knowledge. Knowledge is not just a bunch of nodes with 1:1 connections; there's complex structure in the nodes.