I discovered recently that if you sideload a book onto the Kindle via Calibre, Amazon can delete them silently at some later stage if they decide you didn't purchase it from Amazon. This includes public domain works.
It's allegedly a result of how Calibre manages tags in order to display covers but still feels deeply wrong.
There are two main tag values for Kindle books: EBOK and PDOC.
EBOK is intended for store content. (ebooks)
PDOC is intended for user content. (personal documents)
In the past, only EBOK got cover thumbnails. So Calibre tagged books as ebok to get cover thumbnails.
This changed in a relatively recent firmware update for 7th gen and later Kindles, so now PDOC tagged books get cover thumbnails too. (Sept 2022)
I assume that Amazon just looks at the EBOK tag, assumes its an expired loan or something, and deletes it. Since that thread says only EBOKs get deleted. Calibre still tags as EBOK by default. A possible solution is to tag as PDOC instead. That thread mentions the problem started in 7/2021
m463 · 8h ago
> I assume that Amazon...
I think you should reconsider "the benefit of the doubt".
I remember thinking similar things in the 90's with respect to microsoft.
I was talking to an ex-microsoft coworker about how windows would break compatibility with some 3rd party software. I said "engineers are under pressure to get some feature out, and they don't know how it affects every bit of software out there" or something", giving the benefit of the doubt to microsoft.
But he told me not to be naive, "microsoft would have meetings... How can we control this, how can we own this"
likpok · 11h ago
That’s super annoying, but it is still straightforward to sideload books in a way that works with Amazon sync, via either the email address or uploading through Amazon.
In the Kindle app for Android, Amazon is constantly deleting stuff that you downloaded from it. Why do you think "if they decide you didn't purchase it from Amazon" is relevant?
seam_carver · 7h ago
That sounds like a separate issue than what's he's describing.
Metalnem · 11h ago
I've been a Kindle user for over 15 years, but I finally stopped buying Kindle books after Amazon removed the "Download & Transfer via USB" option, effectively eliminating the ability to remove DRM.
vunderba · 11h ago
It's a shame. I kept my original Kindle (1st gen) around for a long time just so that I could go to "Manage my Content" and then Download the Book via USB for that older device to get an AZW3 file rather than the really locked down KFX format because DeDRM'ing it was easier.
neilv · 12h ago
If you have mixed feelings about Calibre (e.g., powerful, but the GUI looks unpleasant to you), you can use Calibre to help you convert books to EPUB format, then put them in something like a `~/doc/` directory on your laptop, with descriptive filenames.
Then you can read on your laptop, such as with `foliate`, and also sync to an ereader that lets you mount it as USB storage, with a script for which the key part is:
cd ~/doc && rsync -crltv . "${DeviceMountPoint}/."
vorgol · 10h ago
Calibre has improved a lot recently. If you haven't used it for a few years, it's worth taking a fresh look.
beret4breakfast · 12h ago
Or use calibre web. Which has a less functional but nicer looking interface (with user logins)
Or just buy a Kobo and be done with Amazon without any kind of hacking.
goosedragons · 11h ago
Yep. You can also sync to your own self-hosted book platform like Calibre-Web or Komga instead of or alongside the Kobo store. That way you can easily sync books from multiple stores.
dml2135 · 9h ago
I love my Kobo Forma except for the fact that the battery life is complete trash. I’m pretty sure Kobo did not replace the battery on the refurbished device I purchased :(
reaperducer · 11h ago
Or just buy a Kobo and be done with Amazon without any kind of hacking.
Whenever someone on HN complains about Kindles, someone else recommends Kobo. But I never see anyone recommending the Nook.
I haven't used a Nook in close to a decade, but when I did, it was very hackable. I even used a $5 one from Goodwill as an e-ink photo frame. Are the modern Nooks as bad as Kindles now?
tcoff91 · 11h ago
I really like Boox devices. It’s a full on android tablet with e ink.
vorpalhex · 7h ago
Boox is great once you bend it to your will but there is a lot of obscure option mangling to do it. It's not malicious - the boox people are trying to hide the footguns - but there is a lot of "Why doesn't X work?" until you find some buried setting.
tcoff91 · 6h ago
100%. It’s got a learning curve. Especially with the color ones, until you find the dark color enhancement setting,
text is not dark enough.
I opt for Boox because I run the Storyteller app on it, which I also run on my phone so I can easily switch back and forth between listening and reading.
It’s phenomenal once you get it dialed in with storyteller.
msgodel · 11h ago
In my mind if something has any kind of DRM it simply doesn't exist/isn't available.
washmyelbows · 8h ago
so I guess you just don't read ebooks?
sien · 8h ago
There are at least 70K public domain ebooks.
There are sources that say there are 1 M public domain ebooks.
Most ebooks can be found online drm-free extremely easily and/or you can remove the drm yourself.
fumeux_fume · 3h ago
ACSM download, DRM-stripping and uploading to Dropbox can be automated with a python script if you buy from the Kobo store. It requires Calibre, but you just need it for the CLI. Another python library is needed for automating the ACSM auth and download. KOReader's UI is awful, but once you get the hang of it, importing books from Dropbox to read on your Kindle device is pretty nice.
rahimnathwani · 12h ago
If you have epub books you want to read on a Kindle, jailbreaking and installing KOReader seems like overkill.
Why not just convert them to mobi and read them with the native reader?
boneitis · 8h ago
It's a one-time setup and no longer requires conversion.
Granted, it's less of having eliminated a step and more like having shifted the workflow, now having to load into KOReader as the new "default" state if you ever have to reboot the device.
I can put on a custom wallpaper. That you cannot do this without jailbreaking is largely also an ideological/philosophical issue, IMO. And, it's a fun icebreaker if another Kindle user in public walks by and sees a whacky wallpaper. Then, I can share my enthusiasm with hacking or tinkering with computers in general and sometimes that will carry into another line of conversation if everyone's in a conversing mood (of course, without pontificating or breaking out into a lecture about the evils of bigcorp, DRM, etc... I know some people are really bad about that).
Not everyone wants to dump time into tinkering with their Kindle like that, I get it. And honestly, mine collects dust these days, as I find it more difficult to ramp up reading momentum with it, whereas I can more easily (and inadvertently) binge-read if I grab the smartphone thinking I'm just going to squeeze in a few paragraphs, even if the experience is worse.
boneitis · 7h ago
I may have outed myself with "no longer requires conversion". Of course, epubs can have DRM.
WillAdams · 12h ago
Current Kindles will accept .epub files sent via the "Send to Kindle" feature, converting them to be read.
vunderba · 11h ago
I have a pretty old Amazo Kindle Paperwhite (10+ years old) and if you use the send to Kindle email address, you can only send ePub to it, they discontinued support for mobi at some point.
rahimnathwani · 11h ago
OP doesn't want to use Amazon services, so sending files via Amazon's email service isn't relevant.
Which leaves them up to the mercy of Amazon's conversion system. Did they fix it choking on the text encoding yet or is there a decent chance of it messing with apostrophes still?
rahimnathwani · 11h ago
Which leaves them up to the mercy of Amazon's conversion system.
How so?
You can convert the files to mobi using Calibre.
goosedragons · 9h ago
Then you're not using "Send to Kindle"!
rahimnathwani · 11h ago
OP doesn't want to use Amazon services, so 'send to kindle' isn't relevant.
boznz · 11h ago
Until I got my Kobo this was my preferred method. Shame the Kobo does not have such a feature.
It produces EPUB3 files with embedded audio aligned with the text. Use Libation and Calibre to strip your DRM or use Libro.fm to buy audiobooks without DRM.
I am yet to get Whispersync going using 100% amazon products.
tcoff91 · 6h ago
Yeah honestly storyteller works better than Whispersync. I sought this app out because Amazon won’t do whispersync for audiobooks longer than like 44 hrs or something like that. I could not get whispersync for Stormlight archive. Storyteller on the other hand delivered no problem.
xvilka · 12h ago
I wish there was also open and decentralized alternative to Goodreads.
WillAdams · 11h ago
Is Librarything still a thing?
It seemed promising when it launched, but I switched to Goodreads (before the Amazon purchase) and haven't been able to find the time to look for an alternative since.
mikhailt · 11h ago
For what purpose exactly?
There is one based on ActivityPub called BookWyrm.
I have just been checking books out from the library.
thayne · 12h ago
Unfortunately, that doesn't work if your library doesn't have the book you want. And you may have to wait a long time.
If only there was a national (or global) digital library of ebooks.
colonial · 11h ago
> If only there was a ... global digital library of ebooks.
There are several, if the various shadow libraries (Library Genesis, Z-Library, Anna's Archive...) count.
yjftsjthsd-h · 8h ago
Do you not have interloan?
thaumasiotes · 8h ago
> Unfortunately, that doesn't work if your library doesn't have the book you want.
Really? What library are you using? Virtually all of them will arrange to have the book you want sent from a partner library.
thayne · 2h ago
Well, my library does not have any "partner libraries", and in fact, I pay for a library card to a library in a neighboring city, because my city doesn't have its own library.
But I've lived places that had inter-library book loans before, and there were still cases where none of the libraries in the system had a book.
mattkrause · 38m ago
If there’s something you really want/need to get, I would strongly encourage asking a librarian for help (assuming you haven’t already).
I had assumed this would be an imposition and I could handle searching myself. However, good librarians can work magic, whether it’s finding the same material in another format, arranging a loan, or even buying it outright.
The ones I know also assure me this is one of the fun parts of their jobs!
Authors also need to unhook from Amazon, and here are some reasons why:
- They keep at least 30% of the cut, but much more if you dare to include high quality images in your e-books. That map you spend a week or two creating? It'll be a messy blur in a kindle.
- If you are not a best-seller, Amazon brings you nothing in terms of discoverability. In fact:
- the moment a potential reader lands on your book's page, Amazon will show them ads for other books. Consider that that potential reader may have come there after you paid for ads in Amazon or in some other platform, or after you spent a day doing in-person marketing at an arts fair. That's just asinine.
The discoverability issue affects me the most as a reader, since Amz keeps surfacing ten-a-dime stories that happen somewhere in USA, mention starwars three times per page, and involve werewolves.
asdefghyk · 12h ago
RE "...high quality images in your e-books....."
Why is this ? Why are they deliberately converted to low res images ?
Basically, they'll charge you 0.15 USD / Mb if you opt for the 70% royalty. Or nothing, if you opt for the 35% royalty. The deal would sort of make sense if it weren't for because they put so little value on the table.
markush_ · 11h ago
70% is wild, how does that hold up in monopoly and anti competition cases
thaumasiotes · 8h ago
> That map you spend a week or two creating? It'll be a messy blur in a kindle.
There's a funny thing going on with maps in books.
Fantasy novels frequently include maps, and some people are pretty vocal about how much they appreciate those maps. But the maps are never relevant to the story at all; you'd lose absolutely nothing by leaving them out of the book.
History books also tend to include maps. The maps are much less detailed, and it's rare for people to make appreciative comments. But the maps are in there, even though the authors appear not to really care for them, because they're necessary to understanding most of what's going on. As far as relevance to the text is concerned, history books have a much more severe undersupply of maps than fantasy novels have an oversupply, and the fantasy oversupply is considerable.
And yet, somehow, all of the popular demand is for fantasy novels to have more maps and for history books to have fewer.
protocolture · 8h ago
KJ Parker does this really well, he just doesnt include maps, nor does he hold himself accountable to them. Fans trying to stitch his world together run into issues and he just straight up doesnt give a shit.
yjftsjthsd-h · 8h ago
That sounds like the exact opposite of doing it well.
protocolture · 5h ago
How do you figure? You get all the required context from the words in the book.
politelemon · 10h ago
The deascm plugin does work well, but try some different sources for books.
For book deals, try bookbub. There are regularly deals on kobo which you can get notified of.
My first ereader was just a collection of .txt files stored on an ftp server that I would access with my phones web browser.
If I cant get an ebook via Kindle or Google Books its usually available on libgen.
wonderwonder · 11h ago
I've got a nook and always try and buy eBooks through Barnes and noble. Not because I have anything against amazon but because I want physical book stores to survive. Its been a long time since I bought a physical non technical book for myself but I love taking my kids to B&N and letting them roam around and find something to read.
cyberax · 12h ago
I did something similar:
1. Jailbroken my old Kindle to freeze the software version.
2. Use it to get new books from Amazon, and then import them into Calibre. Calibre can then de-DRM them, with the help of the dr-DRM plugin. It only needs the Kindle's serial number to work.
3. In addition to Calibre Desktop, I'm also running Calibre Web with the same database (my book database is on a network drive). Calibre Web also has OPDS server support.
4. I'm now reading books from a reMarkable tablet that has KOReader installed. It also has OPDS support, so I can browse my library from it and download books as needed.
5. I'm also using Storyteller to align Audible books and the Amazon Kindle eBooks, synchronizing them.
Stuff that doesn't work:
1. Position sync between audiobooks and physical books. KOReader does have a position sync protocol, but translating its position into an aligned position is not trivial.
2. Automatic audiobook alignment when new books are added.
3. I'd love to use a Kindle Oasis with 4G with my eSIM to be able to sync the reading position. This was _the_ killer feature of Kindles for me.
It's allegedly a result of how Calibre manages tags in order to display covers but still feels deeply wrong.
Long thread here: https://www.mobileread.com/forums/showthread.php?t=340936
There are two main tag values for Kindle books: EBOK and PDOC.
EBOK is intended for store content. (ebooks) PDOC is intended for user content. (personal documents)
In the past, only EBOK got cover thumbnails. So Calibre tagged books as ebok to get cover thumbnails.
This changed in a relatively recent firmware update for 7th gen and later Kindles, so now PDOC tagged books get cover thumbnails too. (Sept 2022)
I assume that Amazon just looks at the EBOK tag, assumes its an expired loan or something, and deletes it. Since that thread says only EBOKs get deleted. Calibre still tags as EBOK by default. A possible solution is to tag as PDOC instead. That thread mentions the problem started in 7/2021
I think you should reconsider "the benefit of the doubt".
I remember thinking similar things in the 90's with respect to microsoft.
I was talking to an ex-microsoft coworker about how windows would break compatibility with some 3rd party software. I said "engineers are under pressure to get some feature out, and they don't know how it affects every bit of software out there" or something", giving the benefit of the doubt to microsoft.
But he told me not to be naive, "microsoft would have meetings... How can we control this, how can we own this"
Then you can read on your laptop, such as with `foliate`, and also sync to an ereader that lets you mount it as USB storage, with a script for which the key part is:
https://open-slum.org/
Whenever someone on HN complains about Kindles, someone else recommends Kobo. But I never see anyone recommending the Nook.
I haven't used a Nook in close to a decade, but when I did, it was very hackable. I even used a $5 one from Goodwill as an e-ink photo frame. Are the modern Nooks as bad as Kindles now?
I opt for Boox because I run the Storyteller app on it, which I also run on my phone so I can easily switch back and forth between listening and reading.
It’s phenomenal once you get it dialed in with storyteller.
There are sources that say there are 1 M public domain ebooks.
https://libguides.library.umkc.edu/OER/PublicDomainBooks
Also there are thousands and thousands of DRM free ebooks to purchase. Possibly millions.
https://www.tomsguide.com/tablets/e-readers/no-kindle-no-pro...
Why not just convert them to mobi and read them with the native reader?
Granted, it's less of having eliminated a step and more like having shifted the workflow, now having to load into KOReader as the new "default" state if you ever have to reboot the device.
I can put on a custom wallpaper. That you cannot do this without jailbreaking is largely also an ideological/philosophical issue, IMO. And, it's a fun icebreaker if another Kindle user in public walks by and sees a whacky wallpaper. Then, I can share my enthusiasm with hacking or tinkering with computers in general and sometimes that will carry into another line of conversation if everyone's in a conversing mood (of course, without pontificating or breaking out into a lecture about the evils of bigcorp, DRM, etc... I know some people are really bad about that).
Not everyone wants to dump time into tinkering with their Kindle like that, I get it. And honestly, mine collects dust these days, as I find it more difficult to ramp up reading momentum with it, whereas I can more easily (and inadvertently) binge-read if I grab the smartphone thinking I'm just going to squeeze in a few paragraphs, even if the experience is worse.
You can convert the files to mobi using Calibre.
https://storyteller-platform.gitlab.com.io/storyteller/
It has a server and mobile apps for reading.
It produces EPUB3 files with embedded audio aligned with the text. Use Libation and Calibre to strip your DRM or use Libro.fm to buy audiobooks without DRM.
This is the docs site
It seemed promising when it launched, but I switched to Goodreads (before the Amazon purchase) and haven't been able to find the time to look for an alternative since.
There is one based on ActivityPub called BookWyrm.
https://joinbookwyrm.com/
https://openlibrary.org/
If only there was a national (or global) digital library of ebooks.
There are several, if the various shadow libraries (Library Genesis, Z-Library, Anna's Archive...) count.
Really? What library are you using? Virtually all of them will arrange to have the book you want sent from a partner library.
But I've lived places that had inter-library book loans before, and there were still cases where none of the libraries in the system had a book.
I had assumed this would be an imposition and I could handle searching myself. However, good librarians can work magic, whether it’s finding the same material in another format, arranging a loan, or even buying it outright.
The ones I know also assure me this is one of the fun parts of their jobs!
They might be able to also integrate with Hardcover API for syncing your books list as well as using WebDAV protocol for syncing locally book files.
https://github.com/readest/readest
- They keep at least 30% of the cut, but much more if you dare to include high quality images in your e-books. That map you spend a week or two creating? It'll be a messy blur in a kindle.
- If you are not a best-seller, Amazon brings you nothing in terms of discoverability. In fact:
- the moment a potential reader lands on your book's page, Amazon will show them ads for other books. Consider that that potential reader may have come there after you paid for ads in Amazon or in some other platform, or after you spent a day doing in-person marketing at an arts fair. That's just asinine.
The discoverability issue affects me the most as a reader, since Amz keeps surfacing ten-a-dime stories that happen somewhere in USA, mention starwars three times per page, and involve werewolves.
https://kdp.amazon.com/en_US/help/topic/G200634500
Basically, they'll charge you 0.15 USD / Mb if you opt for the 70% royalty. Or nothing, if you opt for the 35% royalty. The deal would sort of make sense if it weren't for because they put so little value on the table.
There's a funny thing going on with maps in books.
Fantasy novels frequently include maps, and some people are pretty vocal about how much they appreciate those maps. But the maps are never relevant to the story at all; you'd lose absolutely nothing by leaving them out of the book.
History books also tend to include maps. The maps are much less detailed, and it's rare for people to make appreciative comments. But the maps are in there, even though the authors appear not to really care for them, because they're necessary to understanding most of what's going on. As far as relevance to the text is concerned, history books have a much more severe undersupply of maps than fantasy novels have an oversupply, and the fantasy oversupply is considerable.
And yet, somehow, all of the popular demand is for fantasy novels to have more maps and for history books to have fewer.
For book deals, try bookbub. There are regularly deals on kobo which you can get notified of.
If I cant get an ebook via Kindle or Google Books its usually available on libgen.
1. Jailbroken my old Kindle to freeze the software version.
2. Use it to get new books from Amazon, and then import them into Calibre. Calibre can then de-DRM them, with the help of the dr-DRM plugin. It only needs the Kindle's serial number to work.
3. In addition to Calibre Desktop, I'm also running Calibre Web with the same database (my book database is on a network drive). Calibre Web also has OPDS server support.
4. I'm now reading books from a reMarkable tablet that has KOReader installed. It also has OPDS support, so I can browse my library from it and download books as needed.
5. I'm also using Storyteller to align Audible books and the Amazon Kindle eBooks, synchronizing them.
Stuff that doesn't work:
1. Position sync between audiobooks and physical books. KOReader does have a position sync protocol, but translating its position into an aligned position is not trivial.
2. Automatic audiobook alignment when new books are added.
3. I'd love to use a Kindle Oasis with 4G with my eSIM to be able to sync the reading position. This was _the_ killer feature of Kindles for me.