Pop out Player/Article to new tab/window
While I do still use services like Amazon and Google for certain movies, books, and comics, I like to retain actual ownership of my digital media whenever possible. In that spirit I run a Plex server in my home network for my collection of movies and music. After using (and loving) a Kindle (Keyboard model) for over 10 years it was finally time for a new eReader. I have tried reading from a computer/tablet screen and definitely notice a difference in eye strain compared to eInk.
For my work and other projects I still hand write a lot of notes. I think the actual act of writing helps cement the information in my head more than typing does, but referring back to or searching old notes becomes difficult with stacks of notebooks. With that in mind I decided to go with an eInk tablet that runs Android and has full stylus note taking with handwriting to text recognition.
Once that decision was made, I started looking into possible changes to my setup for storing/consuming eBooks and Comics. I already had a decent amount of DRM free eBooks from many of the Humble Bundles, and since the new eInk tablet runs Android, it can run the Kindle and Google Play reader apps for any books that you can't find being sold DRM free (Plus there are ways around the DRM of those platforms, but I won't go into that here...). What I wanted was a system similar to Plex that could catalogue, track and serve out my books so that I could consume them in whatever way I wanted (mostly from my eReader, but it is nice to have options from the browser or just to download the file). The most active one I could find was an eBook manager called Calibre, it was open source and seemed to do everything I wanted. In most of the reading I had done, the biggest complaint seemed to be that the web interface was not that great, so another open source project called Calibre-Web was started. Both packages also support the OPDS standard, which a lot of eReading apps use to sync/download eBooks.
It was a little unclear if both packages were needed, so let's find out.
Calibre
My home server is currently still on Ubuntu 18.04 LTS. Luckily this is still supported by Calibre and they have easy install instructions here. Calibre is firstly designed as a desktop app, which I knew may cause some difficulty installing it on a headless system, but it turned out to be an easy process thanks to some documentation I found.
Prerequisites
Calibre requires the xdg-utils, wget, xz-utils, and a version of Python greater than 2.6. I already had a newer version of python and wget, so I installed the other two using the apt command:
sudo apt install xdg-utils sudo apt install xv-utils
Install
Once you have the prerequisites installed, they give you a simple one line install command for the Calibre package:
sudo -v && wget -nv -O- https://download.calibre-ebook.com/linux-installer.sh | sudo sh /dev/stdin install_dir=/opt
There are also options for an isolated install or one for systems without updated CA certificates.
Setup
Calibre is normally used as a desktop app that has a server component. Since I was installing this on a headless server I would not be able to do the setup that way. There is a full command line interface and I found a guide that helped with this process at Digital Ocean. Note that if you watch the video, I mistakenly followed the 14.04 guide so the "xvfb-run" part of the commands are no longer necessary and I have omitted that from the text version here.
The first thing you need some eBooks. If you do not currently have any you can download some from Project Gutenberg, which distributes public domain books. Once you have some you will need to create a calibre database and add your books to it. To do so we can use the calibredb command, it will create a new database/folder structure at the location you specify if one does not already exist. An example of adding a single eBook would be:
calibredb add ~/Downloads/MyBook.epub --with-library /mnt/storage/calibre/
Or you can add a full directory structure of books with the --recurse option:
calibredb add /mnt/storage/Books --library-path /mnt/storage/calibre --recurse
It will add any known file types to your database, and from a basic setup purpose that is all that we need to do.
Automatically Add Books
I kept following the steps from the Digital Ocean tutorial to setup a folder that I could copy new books into, and every 15 minutes the system will check and add any new books into the Calibre database. The crontab entry is as follows:
*/15 * * * * /opt/calibre/calibredb add /mnt/storage/Documents/Books/ToAdd --recurse --library-path /mnt/storage/calibre && rm /mnt/storage/Documents/Books/ToAdd/*
Note that you can also add new books via the web interface, but this is a good way for bulk additions, like when you purchase a new bundle of eBooks.
Web Interface
Calibre comes with its' own web interface. It seems to work well and supports OPDS, but I found it clunky to use and not that nice to look at. It is simple to start with the calibre-server command with the path to your Calibre database:
/opt/calibre/calibre-server /mnt/storage/calibre
You can also make it run as a service, similarly to what we will do with the calibre-web package below.
Calibre-Web
I had come across the calibre-web project while researching and it looked pretty good to me. It is a fork of the Library project which aims to provide a modern web interface to a Calibre database and is Python based. It requires Python 3 and is very easy to install. I did not have the Python3 version of pip (Basically a package manager for Python) so I first installed that:
sudo apt install python3-pip
Then I cloned the calibre-web repository using git:
git clone https://github.com/janeczku/calibre-web
They have a handy file that you can run pip against to install all the prerequisites:
pip3 install --system --target vendor -r requirements.txt
And then you can simply start the server with:
python3 cps.py
When you first browse to the site, it will ask you for the location of your Calibre database, and then you can login with the default admin/admin123 username and password.
Start As Service/daemon
Now that I have confirmed the system is working, I want it to start automatically as a service/daemon. My system is using systemd for daemons so following the instructions from the wiki I created a file called cps.service under /etc/systemd/system with the following:
[Unit] Description=Calibre-Web [Service] Type=simple ExecStart=/usr/bin/python3 /home/lemac/calibre/calibre-web/cps.py WorkingDirectory=/home/lemac/calibre/calibre-web [Install] WantedBy=multi-user.target
Note that for my testing I left the folder underneath my home folder, it may be wise to move this somewhere more global for a final setup. Once that file is created, you can enable the service with the systemctl command and similarly stop/start it as necessary:
sudo systemctl enable cps.service sudo systemctl start cps.service sudo systemctl stop cps.service
Usage/Experience
I found the interface of Calibre-Web to be smoother than the built-in one that comes with Calibre. At first finding books through categories and authors was a bit messy. This seemed to be because the tags it read in from the books themselves were either overly complicated or not present at all, so I took some time going through my collection and cleaning up the tags. The system can fetch info and cover images from online databases which also helped, although I would still cut down the number of tags because I like simpler, more general categories.
A really good feature is the ability to convert your ebooks between a number of different formats (e.g. convert mobi to epub). The conversion leaves the original format intact as well, which helps when you are using different devices/apps to read your books.
The system also allows multiple users, so you can add other people from your household, share your books and allow them to upload theirs. Virtual shelves can also be created to organize your collection however you like, and these shelves can be made public to to other users on your system. For instance I have created public shelves for all comic books, my favourite books, and the books I have finished reading.
Reading
Reading books using this system is pretty easy. From the browser itself you can read any book in a supported format, and it will keep track of your reading progress. You can also download the file in any of the available formats to read on your computer, or copy over to your reader/tablet via USB or other means.
I prefer doing my reading on eInk screens. I spend so much of my time in front of brightly lit screens, so reading on eInk definitely is more comfortable on my eyes. For my old Kindle (Or if you use the Kindle app), Calibe-Web supports the send to kindle option as long as you have an smtp/mail server setup. This makes it as easy as clicking on the book, and then clicking the "Send Mobi to Kindle" option. The next time you turn on your Kindle, the book will be downloaded.
For my new Onyx Boox device, I wanted to try out the OPDS support that Calibre-Web has, so I started using the Moon+ Reader app. Inside the app you simply create a new "Net Library" and point it to the address of your Calibre-Web server with /opds added to the end. You can then browse through all categories and shelves you have created, select a book you want to read, and tap "Download". The book downloads to your device and you can start reading.
Finding DRM free eBooks and Comics
To get more books into your collection, they will need to be DRM free, which can be a little more inconvenient but not too bad. There are a bunch of publishers/authors who have specified that their books be sold without DRM. If you have a particular store you buy from, do a search to see if they have a DRM free section. Also, the Humble book and comic bundles are generally all DRM free. Here are some links to DRM free sources I have found:
- Ebooks.com DRM free section (store) - https://www.ebooks.com/en-ca/drm-free/
- Project Gutenberg (Free public domain ebooks) - https://www.gutenberg.org/
- Comixoly DRM free section (Comics store) - Link
- Current Humble Books/Comics bundes - https://www.humblebundle.com/books
There are also ways to get your current Kindle collections and other DRM protected books into Calibre, but I will not cover that here.
ToDo
That is basically it for a setup of my new book server. There are some things I still want to tweak, and a couple decisions to make. I want to add a certificate and make the service run over SSL so that it is encrypted. I could possibly move this setup to my online host, so that my collection is available from anywhere, but I think I will keep it local as it is a rare occasion I am out somewhere and need to download a book.
As I said before I am calling these articles "How Did I"s instead of "How To"s because I am by no means an expert, and would like these articles to encourage people to give these things a try and let me know how I could improve my processes. You can participate via the message boards on the BBS, the Discord server (invite link on your user dashboard), via Twitter, or through the comments on the video available on Odysee/LBRY and YouTube.
References
- Calibre - https://calibre-ebook.com/about
- Calibre-Web - https://github.com/janeczku/calibre-web
- OPDS - https://en.wikipedia.org/wiki/Open_Publication_Distribution_System
- Digital Ocean Guide: Link