"Instant Loading" with IndexedDB (Building a PWA, Part 2)

18 October 2016

Last week's article was Part 1 (of 3) in my series on how I built a Progressive Web Application for this blog.

The application

As I mentioned, there are two parts to creating an "offline-first" application -

Cache the app shell so pages and assets are accessible offline
Load locally-saved data first, then fetch updated data later if the network permits

Last week, I showed how I achieved the first using Service Worker to cache the app shell and serve it to the user even when they are offline. In this article, I will show how I used IndexedDB to save the fetched data for offline use. In the final part of this series next week, I will demonstrate how I implemented push notifications.

What is IndexedDB? #

IndexedDB is a "low-level API for client-side storage of significant amounts of structured data" (Mozilla). It is a JavaScript-based, object-oriented, database that allows us to easily store and retrieve data that has been indexed with a key. IndexedDB can take some getting used to as it very low-level and works quite differently to other databases.

For the bitsofcode web app, I used IndexedDB to store the articles fetched from the blog RSS feed. This way, articles could be loaded instantly from the database first, before checking over the network for the more up-to-date data.

Creating a Database and Store #

For my own sanity (and because I love Promises), I use Jake Archibald’s IndexedDB Promised library which offers a Promise-ified version of the IndexedDB methods. All my examples will be using this library.

To start using IndexedDB, we need to create our database and the stores (which are like tables) within the database. To do that, we need to open up IndexedDB with the database name and version.

idb.open('DATABASE_NAME', VERSION, function(upgradeDb) {
	// Create stores
});

To create stores in our database, we use the createObjectStore method of upgradeDb. When creating a store, we pass two arguments -

The name of the store
An object with some optional settings. For example, the keyPath property, which allows us to specify what key in the object we want to be the primary key.

Here it is all together -

idb.open('DATABASE_NAME', VERSION, function(upgradeDb) {
	const store = upgradeDb.createObjectStore('STORE_NAME', {
    	keyPath: 'PRIMARY_KEY_NAME'
    });
});

We can also define indexes for the stores. Indexes provide us alternative ways of accessing the data. For example, in a store we may have data that has, amongst others, a timestamp and name key. If we ever want to retrieve the data by timestamp or name, we may want to have indexes for each of these keys.

idb.open('DATABASE_NAME', VERSION, function(upgradeDb) {
	const store = upgradeDb.createObjectStore('STORE_NAME', {
    	keyPath: 'PRIMARY_KEY_NAME'
    });
    store.createIndex('INDEX_NAME', 'KEY_PATH');
});

For the bitsofcode application, I created 3 stores - an ArticlesStore, a BookmarksStore and a SettingsStore. Here is how I created the first.

idb.open('bitsofcode', 1, function(upgradeDb) {
	const ArticlesStore = upgradeDb.createObjectStore('Articles', {
    	keyPath: 'guid'
    });
    ArticlesStore.createIndex('guid', 'guid')

    // Other stores..
});

This process of opening up IndexedDB and creating/updating the database itself returns the database, to which we can create/read/update/delete data. Because of this, it is best to wrap the creation of the database and stores in a function that can be reused.

const OpenIDB = function() {
	return idb.open('bitsofcode', 1, function(upgradeDb) {
	    const ArticlesStore = upgradeDb.createObjectStore('Articles', {
    	    keyPath: 'guid'
        });
        // etc etc
    });
};

Data CRUD #

Once the database and stores have been created, we can move on the the CRUD actions.

Create / Update #

IndexedDB is a transactional database system, which is a system where operations on the database can be rolled back if not completed properly. This means that whenever we want to interact with any of our data, we need to first open up a new transaction, perform our actions, then either commit or rollback the transaction depending on if things succeed. This may seem like a lot of tedious work to do, but this is critical for maintaining the integrity of data.

Therefore, to create (or update) data in our IndexedDB, we need to do the following things -

Open up the database
Open a new read/write transaction with the store within the database
Add the data to the store
Complete the transaction

These steps looks like this -

// 1. Open up the database
OpenIDB().then((db) => {
	const dbStore = 'ArticlesStore';

	// 2. Open a new read/write transaction with the store within the database
	const transaction = db.transaction(dbStore, 'readwrite');
	const store = transaction.objectStore(dbStore);

	// 3. Add the data to the store
	store.put(data);

	// 4. Complete the transaction
	return transaction.complete;
})

We are using the store.put() method here, which is also used for updating data in the database.

Read #

To retrieve data from IndexedDB, we need to do the following things -

Open up the database
Open a new read-only transaction with the store within the database
Return the data

These steps looks like this -

// 1. Open up the database
OpenIDB().then((db) => {
	const dbStore = 'ArticlesStore';

	// 2. Open a new read-only transaction with the store within the database
	const transaction = db.transaction(dbStore);
	const store = transaction.objectStore(dbStore);

	// 3. Return the data
	return store.getAll();
})

We can also get data by a specific index -

// 1. Open up the database
OpenIDB().then((db) => {
	const dbStore = 'ArticlesStore';

	// 2. Open a new read-only transaction with the store within the database
	const transaction = db.transaction(dbStore);
	const store = transaction.objectStore(dbStore);

    const index = store.index('INDEX_NAME');

	// 3. Return the data
	return index.getAll();
})

We can even get data by a value that correlates to the index -

// 1. Open up the database
OpenIDB().then((db) => {
	const dbStore = 'ArticlesStore';

	// 2. Open a new read-only transaction with the store within the database
	const transaction = db.transaction(dbStore);
	const store = transaction.objectStore(dbStore);

    const index = store.index('INDEX_NAME');

	// 3. Return the data
	return index.getAll('VALUE');
})

Delete #

Finally, to delete data from IndexedDB, we need to do the following things -

Open up the database
Open a new read/write transaction with the store within the database
Delete the data corresponding to the passed key
Complete the transaction

These steps looks like this -

// 1. Open up the database
OpenIDB().then((db) => {
	const dbStore = 'ArticlesStore';

	// 2. Open a new read/write transaction with the store within the database
	const transaction = db.transaction(dbStore, 'readwrite');
	const store = transaction.objectStore(dbStore);

	// 3. Delete the data corresponding to the passed key
	store.delete('DATA_KEY');

	// 4. Complete the transaction
	return transaction.complete;
})

Instant Loading Data with IndexedDB #

By saving the articles locally in IndexedDB, I was able to first serve those locally saved articles to users first, before going to the network to get the most up-to-date data. This is what the sequence of functions look like for the Latest Articles page.

Database.retrieve('Articles')
        .then((articlesFromDatabase) => {
            if (articlesFromDatabase.length == 0) return fetchArticles(true)
            didFetchArticlesFromDatabase = true;
            return Promise.resolve(articlesFromDatabase);
        })
        .then((articles) => {
            Articles = sortedArticles(articles);
            displayArticles(Articles);
            return Promise.resolve();
        })
        .then(() => { if (didFetchArticlesFromDatabase) updateArticlesInBackground() });