Project idea from the bottom of a drawer

You know that promise you make to yourself every so often about building that project you came up with years ago? Everyone owns a virtual drawer filled with ideas for amazing projects. Have you ever actually built something from it? Me neither, until a few days ago.

My family and I own a book collection. A rather large book collection. Most of us like reading, so it makes sense to own lots of books. One day, a few years ago, a thought crossed our minds: we should catalog every book we own. And that’s what we did. Well, my father did. He decided to go through the hard task of picking up every book we have out of several bookshelves and write on an Excel file the title, the author, the genre, the publisher, and the year. He did that for every 340 books we owned at that time.

With that Excel file, we could see every book we had, with the added benefit of giving us the ability to use the search and filtering features that Excel natively provides. This was useful at some point, but having that Excel file in an old desktop computer at home wasn’t enough. My main issue was always being afraid of buying a book we already had, and that Excel file couldn’t help me. So I needed to put that catalog to some better use.

A few years ago, I imported the catalog’s data into a MongoDB and made it accessible in a Laravel app. It kind of worked, in the sense that I could access remotely my family’s book catalog. Although, I was dumb enough to forget to push that code to a Git repo, so I eventually lost everything in a server migration. Back to square one.

We’re on the holidays and I found myself with a few days free. I had some web stacks that I’d like to learn or revisit, so I finally decided to pick up the book catalog idea from the bottom of the drawer.

I started with the backend side of things. Decided to stick with MongoDB because the data wouldn’t be relational anyway, so Mongo should do the job with no issues. For the API, I went with Koa. To this day, I’m most familiar with Javascript for both backend and frontend. I have used some backend Javascript frameworks in the past (Loopback, ExpressJS), and Koa was recommended by a former client in a project here at Runtime. Setting up Koa is pretty similar to ExpressJS, but what I liked the most about Koa was having support for async/await functions (so no Node style callback() hell — yay! — among other things). Also, since I didn’t need view rendering or a ton of middlewares, Koa’s modularity just felt right.

After setting up a MongoDB Node ODM and a Goodreads API connection (from where we’ll fetch book data), the API was done.

Time to get to the client side. I wanted this to be simple. Just a simple, paginated list with the books data was enough. Maybe some search features. But nothing more than that.

I’ve been wanting to try Next.js for some time, since I have been reading about it on Twitter and didn’t know what it was. Next.js is a React framework that allows for Server Side Rendering (among other features) that, allegedly, makes it blazing fast. Also, it does File-System Routing so that should make my life just a little easier.

And you know what? For a React developer, it is the exact same thing. The whole way through I felt just as if I were coding a React app. It is super simple and almost everything works the same way (hooks included). So, after some reading on how Next.js works, and adding Bulma.io for some styling, I had my client app up and running.

Everything done, right? Kind of. I still needed to make it available on the Internet. I’ve done that in the past so that should be a breeze.

I have a small Ubuntu server on Linode that I use to host this kind of small projects, so after pulling the apps to the server, installing MongoDB, setting up NGINX to proxy_pass to the Node apps, and setting up DNS on Cloudflare, everything was online.

Tip: I use PM2 to manage my Node apps. It’s a process manager that relaunches the apps if anything goes wrong, with minimal setup.

And that’s it. My book catalogue was live. And empty. Which has no use whatsoever. And that needs to be fixed.

I had a problem. On one hand there’s a book catalog waiting to be used, and on the other hand there’s an Excel file. That file contained titles and authors for 340 books. But it was missing an important piece of the puzzle: the ISBN.

The International Standard Book Number is a 13-digit numeric commercial book identifier which is intended to be unique (in Wikipedia), and that was the best way to assert that there were no duplicates in the database. Since I didn’t have the ISBNs, I was afraid my data would be inconsistent.

To fix this, I considered three approaches:

1. Import the whole Excel file into the database and have those books have null ISBNs, but the data wouldn’t be consistent and that usually makes my eyes twitch.
2. Write a script that reads the Excel file, search the Goodreads API for the title of the book, cross my fingers and hope for a result that would match the search perfectly. Spoiler alert: It doesn’t work. Most of the times, the Goodreads API wouldn’t return a best match for what I was searching, unless I searched with ISBN (which I didn’t have). Also, the Goodreads API throttling is inconsistent at best, so unless I had built a bulletproof script, going with this solution would be pretty much annoying and time consuming.
3. Catalog everything again.

Believe it or not, I went with option three. Cataloging everything again should be easy, since now we only need the ISBNs. So I built a page on my Next.js app with an input and a button to submit a ISBN to the API. Then the API would fetch the data from Goodreads and insert it into the database. But then I remembered that inserting a 13-digit-long number 340 times would be a pain. So I went the extra mile.

Another good thing about ISBNs is that they’re in every book in barcode form. So I only needed a barcode scanner to scan every book on our bookshelves.

I’ve built mobile apps before. I’ve built the iOS and Android apps for Suprimidos.pt and Fogos.pt using Flutter and it is my go to solution for building quick and easy mobile apps with minimal effort. I’ve written about it before (here and here) so take a look if you’re curious about it.

This was the perfect use case for a Flutter app. In less than half an hour I had put together an app that scans the barcode and makes a request to the API. I know, it’s a fairly simple app. But it does the job and works on Android and iOS devices. I didn’t even bother publishing it to the app stores. I just sideloaded the app into everyone’s phones and I was good to go.

So, in a couple of days, using 2 to 3 hours a day, I built a full stack personal book catalog. That’s a pretty hefty idea to take out of my drawer. And you? What projects have you been postponing forever?

Happy New Year 2020.

I love building products and I found my place to do so at Runtime Revolution. If you are interested in who we are and what we do, feel free to reach out!