gwagner's blog

CRUD Operations with IndexedDB - Part 1: Creating and Clearing a Database

gwagner — Mon, 12 Jul 2021 15:02:47 +0000

In this article, we show how to create and clear an IndexedDB database keeping all code in a single HTML file. Creating an IndexedDB database means first creating an empty database, and then adding records to its empty tables (or "object stores"). In Part 2, we will explain how to retrieve, update and delete IndexedDB records, such that we get the full collection of CRUD operations: Create, Update, Retrieve and Delete.

This is not an introduction to IndexedDB, but rather a guide how to use it for CRUD operations in a front-end app. We assume that the reader has already done an introductory tutorial such as Working with IndexedDB. Recall that an IndexedDB database is a set of object tables (or "object stores") where each table row, or record, has a standard ID (or "primary key") property defined with keyPath when creating the database.

As recommended in Working with IndexedDB, we use the IndexedDB Promised library, which wraps the indexedDB API with ES6-Promise/await-based methods for obtaining more readable and maintainable code in our IndexedDB access methods. Download it into your project folder such that it can be included with <script src="idb.js"></script> or load it from the CDN https://unpkg.com/idb/build/iife/index-min.js.

We extend the "idb" namespace variable by adding three database management methods:

idb.createEmptyDB( tableNames): Checks if there is already a database with the given name, and, if not, creates one with an empty table (or "object store") for each of the table names provided.
idb.addRecords( tableName, records): Adds a list of records to the given table.
idb.clearDB(): Clears the contents of all tables.

We start with looking at the code of the createEmptyDB method:

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
 <meta charset="UTF-8" />
 <title>CRUD Operations with IndexedDB - Part 1</title>
 <!-- Load the IndexedDB Promised library -->
 <script src="idb.js"></script>
 <script>
  const dbName = "test";
  /*******************************************************************
   * Create an empty IndexedDB
   *******************************************************************/
  idb.createEmptyDB = async function (dbName, tableNames) {
   idb.db = await idb.openDB( dbName, 1, {
    upgrade(db) {
     for (const tn of tableNames) {
      if (!db.objectStoreNames.contains( tn)) {
       db.createObjectStore( tn, {keyPath:"id"});
      }
     }
    }
   });
  };

We define a global variable dbName as the name of our IndexedDB database. The idb.createEmptyDB method takes a list of table names and tries to open the database (with version number 1). If this fails, that is, if the database does not yet exist, the function with the paramter upgradeDb will be called, taking care of creating an empty object store with a standard ID property "id" for each of the table names provided.

For adding a list of records to a table (= object store) in one (transactional) step, we invoke the IndexedDB add method on the corresponding object store for each of them in parallel. Since each add invocation returns a promise object, we create a list of promise objects by mapping the records array to an array of add invocation expressions and use the Promise.all method for resolving this list of promises only when all of them have resolved:

  idb.addRecords = async function (tableName, records) {
    // create a transaction involving only the table with the provided name
    const tx = idb.db.transaction( tableName, "readwrite");
    // create a list of add invocation expressions
    const addInvocationExpressions = records.map( r => tx.store.add( r));
    // invoke all of them in parallel and wait for their completion
    await Promise.all( addInvocationExpressions);
    // wait for the completion of the transaction tx
    await tx.done;
  };

For clearing the entire database contents, we need to clear all object tables/stores by invoking the IndexedDB clear method on them. Again, we create a list of promises by mapping the idbCx.objectStoreNames collection to an array of clear invocation return values and use the Promise.all method for resolving this list only when all of them have resolved:

  idb.clearDB = async function (dbName) {
    // only open DB and store a connection reference if not yet done
    idb.db ??= await idb.openDB( dbName);
    // create a transaction involving all tables of the database
    const tx = idb.db.transaction( idb.db.objectStoreNames, "readwrite");
    // create a list of clear invocation expressions
    const clearInvocationExpressions =
        Array.from( idb.db.objectStoreNames, osName => tx.objectStore( osName).clear());
    // invoke all of them in parallel and wait for their completion
    await Promise.all( clearInvocationExpressions);
    // wait for the completion of the transaction tx
    await tx.done;
  };

We also define a createTestData method:

  async function createTestData () {
   await idb.addRecords( "books", [
     {id: "006251587X", title: "Weaving the Web", year: 2000, edition: 2},
     {id: "0465026567", title: "Gödel, Escher, Bach", year: 1999},
     {id: "0465030793", title: "I Am a Strange Loop", year: 2008}
   ]);
  }

Finally, we define an immediately invoked function expression (IIFE) for creating an empty database with name "test" having an empty table "books" and HTML buttons for invoking the createTestData method and for invoking the clearDB method:

  (async function () {
   await idb.createEmptyDB("test",["books"]);
  })();
  </script>
</head>
<body>
 <h1>Working with IndexedDB</h1>
 <h2>Creating and Clearing a Database</h2>
  <ul>
    <li><button type="button" onclick="createTestData()">Generate</button> test data</li>
    <li><button type="button" onclick="idb.clearDB()">Clear</button> database tables</li>
  </ul>
</body>
</html>

Now it's time to try this out. Copy and paste all 5 HTML/JS code fragments from this article to a text editor and save it as an HTML file. Then, open it in a browser that allows accessing IndexedDB databases via file URLs (see the note below). Otherwise, open it via a (possibly local) web server. You can inspect the existence and contents of IndexedDB databases with the help your browser's developer tools (e.g. F12 or with Shift-Ctrl-I). Try out the following steps:

Check if an IndexedDB database with name "testDB" and an empty "books" table has been created.
Press the button.
Then check if the "books" table has been populated with 3 book records.
Press the button again and observe a constraint validation error (in the console) due to an attempt to create another record with the same ID as an already stored one.
Press the button.
Then check if the "books" table has been cleared.
Now you could create the test data again.

Attention

While you can directly run IndexedDB JS code from a (local or remote) website (e.g., from a GitHub Pages website), you can only run it from your local file system after changing your browser's default configuration.

For FireFox, you have to set the configuration property security.fileuri.strict_origin_policy to false by first entering about:config in the browser's web address bar and then entering this property name.

Asynchronous Programming in JavaScript

gwagner — Thu, 01 Jul 2021 12:36:44 +0000

In programming, we often have the situation that, when calling a possibly time-consuming input/output (I/O) operation (or any long-running operation, e.g., for performing a complex computation), the program execution has to wait for its result being returned before it can go on. Calling such an operation and waiting for its result, while the main program's further execution (and its entire thread) is blocked, represents a synchronous operation call. The implied waiting/blocking poses a problem for a JS program that is being executed in a browser thread since during the waiting time the user interface (in a browser tab) would be frozen, which is not acceptable from a usability point of view and therefore not accepted by browsers.

Consequently, in JavaScript, it is not possible to call an I/O operation, e.g., for fetching data from a webpage (with the built-in XMLHttpRequest or fetch API) or for accessing a remote database (via HTTP request-response messaging) synchronously. These types of operations have to be performed in an asynchronous (non-blocking) manner, instead.

Asynchronous programming concepts in JavaScript have undergone an evolution from callbacks to promises to generators (coroutines) and, most recently, to asynchronous procedure calls with await procedure invocation expressions and asynchronous procedure definitions with async. Each evolution step has made asynchronous programming a little bit easier for those who have taken the effort to get familiar with it.

Due to this evolution, operations of older JS input/output APIs available in the form of built-in objects, like XMLHttpRequest for HTTP messaging or indexedDB for object database management, work with callbacks, while newer APIs, like fetch for HTTP messaging, work with promises and can also be invoked with await.

Callbacks

A simple asynchronous programming approach consists of defining a procedure that is to be executed as soon as the asynchronous operation completes. This allows to continue the program execution after the invocation of the asynchronous operation, however, without assuming that the operation result is available. But how does the execution environment know, which procedure to call after completing the asynchronous operation?

In JS, we can pass a JS function as an argument in the invocation of the asynchronous operation. A callback is such a JS function.

Consider the following example. An external JS file can be dynamically loaded (in the context of an already loaded webpage with associated JS code) by (1) programmatically creating an HTML script element DOM object with the file's URL as the value of the script's src attribute, and (2) inserting the newly created script element after the last child node of the document's head element:

function loadJsFile( fileURL) {
  const scriptEl = document.createElement("script");
  script.src = fileURL;
  document.head.append( scriptEl);
}

When the new script element is inserted into the document's DOM, e.g., with the help of the asynchronous DOM operation append (at the end of the loadJsFile procedure), the browser will load the JS file and then parse and execute it, which will take some time. Let's assume that we have a JS code file containing the definition of a function addTwoNumbers that does what its name says and we first load the file and then invoke the function in the following way:

loadJsFile("addTwoNumbers.js");
console.log( addTwoNumbers( 1, 2));

This wouldn't work. We would get an error message instead of the sum of 1 and 2, since the intended result of the first statement, the availability of the addTwoNumbers function, is not (yet) obtained when the second statement is executed.

We can fix this by adding a callback procedure as a second parameter to the loadJsFile procedure and assign it as an event handler of the JS file load event :

function loadJsFile( fileURL, callback) {
  const scriptEl = document.createElement("script");
  script.src = fileURL;
  script.onload = callback;
  document.head.append( scriptEl);
}

Now when calling loadJsFile we can provide the code to be executed after loading the "addTwoNumbers.js" file in an anonymous callback function:

loadJsFile("addTwoNumbers.js", function () {
  console.log( addTwoNumbers( 1, 2));  // results in 3
]);

Since the loading of the JS file can fail, we should better add some error handling for this case by defining an event handler for the error event. We can handle possible errors within the callback procedure by calling it with an error argument:

function loadJsFile( fileURL, callback) {
  const scriptEl = document.createElement("script");
  script.src = fileURL;
  script.onload = callback;
  script.onerror = function () {
      callback( new Error(`Script load error for ${fileURL}`));
  };
  document.head.append( scriptEl);
}

Now we call loadJsFile with an anonymous callback function having an error parameter:

loadJsFile("addTwoNumbers.js", function (error) {
  if (!error) console.log( addTwoNumbers(1,2));  // results in 3
  else console.log( error);
]);

Callbacks work well as an asynchronous programming approach in simple cases. But when it is necessary to perform several asynchronous operations in a sequence, one quickly ends up in a "callback hell", a term that refers to the resulting deeply nested code structures that are hard to read and maintain.

Promises

A promise (also called future in some programming languages, like in Python) is a special object that provides the deferred result of an asynchronous operation to the code that waits for this result. A promise object is initially in the state pending. If the asynchronous operation succeeds (in the case when the resolve function is called with an argument providing the result value), the promise state is changed from pending to fulfilled. If it fails (in the case when the reject function is called with an argument providing the error), the promise state is changed from pending to rejected.

An example of a built-in asynchronous operation that returns a promise is import for dynamically loading JS code files (and ES6 modules). We can use it instead of the user-defined loadJsFile procedure discussed in the previous section for loading the addTwoNumbers.js file and subsequently executing code that uses the addTwoNumbers function (or reporting an error if the loading failed):

import("addTwoNumbers.js")
.then( function () {
  console.log( addTwoNumbers( 1, 2));
})
.catch( function (error) {
  console.log( error);
});

This example code shows that on the promise object returned by import we can call the predefined functions then and catch:

then: for continuing the execution only when the import operation is completed with a fulfilled promise, and
catch: for processing the error result of a rejected promise.

The general approach of asynchronous programming with promises requires each asynchronous operation to return a promise object that typically provides either a result value, when the promise is fulfilled, or an error value, when the promise is rejected. For user-defined asynchronous procedures, this means that they have to create a promise as their return value, as shown in the promise-valued loadJsFile function presented below.

A promise object can be created with the help of the Promise constructor by providing an anonymous function expression as the argument of the Promise constructor invocation (with two parameters resolve and reject representing JS functions). We do this in the following example of a promise-valued loadJsFile function, which is a variant of the previously discussed callback-based loadJsFile procedure:

function loadJsFile( fileURL) {
  return new Promise( function (resolve, reject) {
    const scriptEl = document.createElement("script");
    scriptEl.src = fileURL;
    scriptEl.onload = resolve;
    scriptEl.onerror = function () {
        reject( new Error(`Script load error for ${fileURL}`));
    };
    document.head.append( scriptEl);
  });
}

This new version of the asynchronous loadJsFile operation is used in the following way:

loadJsFile("addTwoNumbers.js")
.then( function () {
  console.log( addTwoNumbers( 1, 2));
})
.catch( function (error) {
  console.log( error);
});

We can see that even the syntax of a simple promise-valued function call with then and catch is more clear than the syntax of a callback-based asynchronous procedure call. This advantage is even more significant when it comes to chaining asynchronous procedure calls, as in the following example where we first sequentially load three JS files and then invoke their functions:

loadJsFile("addTwoNumbers.js")
.then( function () {
  return loadJsFile("multiplyBy3.js");})
.then( function () {
  return loadJsFile("decrementBy2.js");})
.then( function () {
  console.log( decrementBy2( multiplyBy3( addTwoNumbers(1,2))));})
.catch( function (error) {
  console.log( error);
});

Notice that for executing a sequence of asynchronous operations with then, we need to make sure that each then-function returns a promise.

As an alternative to the sequential execution of asynchronous operations, we may also execute them in parallel with Promise.all:

Promise.all([ loadJsFile("addTwoNumbers.js"),
  loadJsFile("multiplyBy3.js"),
  loadJsFile("decrementBy2.js")
])
.then( function () {
  console.log( decrementBy2( multiplyBy3( addTwoNumbers(1,2))));
})
.catch( function (error) {console.log( error);});

Unlike loadJsFile, which simply completes with a side effect (the loading of JS code), but without a result value being returned, a typical asynchronous operation returns a promise object that provides either a result value, when the promise is fulfilled, or an error value, when the promise is rejected.

Let's consider another example, where we have asynchronous operations with result values. The JS built-in fetch operation allows retrieving the contents of a remote resource file via sending HTTP request messages in two steps:

In the first step, it returns a promise that resolves with a response object as its result value containing the HTTP header information retrieved.
Then, invoking the text() or the json() function on the previously retrieved response object returns a promise that resolves to the HTTP response message's body (in the form of a string or a JSON object) when it is retrieved from the remote server.

In such a case, when we chain two or more asynchronous operation calls with result values, each successor call can be expressed as a transformation from the previous result to a new result using arrow functions as shown in line 2 of the following example:

fetch("user1.json")
.then( response => response.json())
.then( function (user1) {alert( user1.name);})
.catch( function (error) {console.log( error);});

Notice that the text file "user1.json" is assumed to contain a JSON object describing a particular user with a name field. This JSON object is retrieved with the arrow function expression in line 2.

Calling asynchronous operations with `await`

When a program with a statement containing an asynchronous procedure call (with await) is executed, the program will run up to that statement, call the procedure, and suspend execution until the asynchronous procedure execution completes, which means that if it returns a Promise, it is settled. That suspension of execution means that control is returned to the event loop, such that other asynchronous procedures also get a chance to run. If the Promise of the asynchronous procedure execution is fulfilled, the execution of the program is resumed and the value of the await expression is that of the fulfilled Promise. If it is rejected, the await expression throws the value of the rejected Promise (its error).

When we use await for invoking a Promise-valued JS function, we typically do not use Promise chaining with .then, because await handles the waiting for us. And we can use a regular try-catch block instead of a Promise chaining .catch clause, as shown in the following example code:

try {
  await loadJsFile("addTwoNumbers.js");
  console.log( addTwoNumbers(2,3));
} catch (error) {
  console.log( error);
}

Notice that this is the code of an ES6 module. In a normal JS file, await can only be used within async functions.

When we call several asynchronous procedures in succession with await, the code reads in a natural way, similar to the code for calling synchronous procedures:

try {
  await loadJsFile("addTwoNumbers.js");
  await loadJsFile("multiplyBy3.js");
  await loadJsFile("decrementBy2.js");
  console.log( decrementBy2( multiplyBy3( addTwoNumbers(2,3))));
} catch (error) {
  console.log( error);
}

In an async function, we can invoke Promise-valued functions in await expressions. Since an async function returns a Promise, it can itself be invoked with await.

async function load3JsFiles() {
  await loadJsFile("addTwoNumbers.js");
  await loadJsFile("multiplyBy3.js");
  await loadJsFile("decrementBy2.js");
}
try {
  await load3JsFiles();
  console.log( decrementBy2( multiplyBy3( addTwoNumbers(2,3))));
} catch (error) {
  console.log( error);
}

In the more typical case of asynchronous operation calls with result values, we obtain code like the following await-based version of the above promise-based example of using fetch:

try {
  const response = await fetch("user1.json");
  const user1 = await response.json();
  alert( user1.name);
} catch (error) {
  console.log( error);
}

For more about asynchronous programming techniques, see Promises, async/await and Demystifying Async Programming in Javascript.

The Most Important New JavaScript Elements for Basic Programming

gwagner — Tue, 21 Apr 2020 21:16:48 +0000

There have been many articles about the new language elements introduced to JavaScript by ECMAScript 2015+ (that is, 2015+16+17+18+19+20+...). Typically, in these articles, a great number of new features are presented without taking their relevance into consideration, often with artificial programming examples that do not help to exhibit the potential use of a new feature.

In this post, we only present the most important new language elements, which are generally useful for all kinds of basic programming tasks, in a summarized way.

Block-Scope Variable Declarations with `let` and `const`

ES5 did not allow declaring variables the scope of which is a block delimited by a pair of curly braces, { and }, or defined by a for loop. Rather, all variables declared with var, even if declared within a block, have either a function scope or the global scope. The new feature of block-scope variable declarations with let and const allows declaring local variables with a finer-grained scope, which helps avoiding unintended variable duplications and leads to more modular code.

There is only one meaning difference between let and const. Variables declared with const are frozen or immutable, and in this sense constant, while let variables are not. It is preferable to use const for all variables that are not supposed to change their values. Otherwise, or if this is not clear, one should declare the variable with let if it is a block-scoped variable, or with var if it is a global or function-scoped variable.

Arrow Functions with `=>`

Arrow functions provide a more concise function expression syntax, see Ex. 1, and allow using JavaScript's this variable from the function’s outer environment (its closure) in their function body, see Ex. 2.

Example 1:

let evens = [0,2,4,6,8];
// instead of evens.map( function (v) {return v+1;})
let odds = evens.map( v => v+1);  // [1,3,5,7,9]

Example 2:

this.nums = [1,3,5,8,10,12,15,17];
this.fives = [];
// instead of this.nums.forEach( function (v) {if (v % 5 === 0) this.fives.push(v);}, this)
this.nums.forEach( v => {if (v % 5 === 0) this.fives.push(v);});

For-Of Loops over Iterable Objects

Iterable objects include strings, arrays, array-like objects (e.g., the built-in arguments object or instances of HTMLCollections and NodeList), and instances of the datatype objects TypedArray, Map, and Set, as well as user-defined iterables. For instance,

const divElems = document.getElementsByTagName("div");  // an HTMLCollection
for (let dEl of divElems) {
  console.log( dEl.id);
}

A for-of loop is often more handy than a for loop whenever a counter variable is not needed. As opposed to a forEach loop, a for-of loop allows iterating over HTMLCollections and can be abandoned with break.

Template Literals

... are enclosed by backtick characters (like `... `) instead of double or single quotes and allow a concise syntax for (possibly multi-line) string values resulting from a combination of fixed text parts and variables/expressions. For instance,

const classValues = "card important";
const name = "Joker";
const htmlTemplate = `<div class="${classValues}">
  <p>Hello ${name}!</p>
</div>`

The Spread Operator `...`

allows spreading (1) the elements of an iterable collection in places where arguments for function calls or array elements are expected, or (2) the slots of a JS object in places where name-value pairs are expected. For instance,

let nums = [3,4,5], otherNums = [1, 2, ...nums];  // [1,2,3,4,5]
// cloning an array
let numsClone = [...nums];
// cloning an object
let book = {title:"JavaScript: The Good Parts"};
let bookClone = {...book};

Destructuring Assignments

Allow a concise syntax for assigning the property values of a JS object or the elements of a JS array to corresponding variables. For instance,

var point1 = [10,5];
var [x,y] = point1;
console.log(`x = ${x} | y = ${y}`);  // x = 10 | y = 5

var person1 = {firstName:"James", lastName:"Bond"};
var {first, last} = person1;
console.log(`first:${first} | last:${last}`);  // first:"James" | last:"Bond"

Use Case 1: Dealing with multiple return values of a function.

function getRectangle () {
  return {width: 50, height: 20};
}
const {a, b} = getRectangle();
drawRectangle( a, b);

Use Case 2: Swapping two variables.

var a = 1, b = 2;
[a,b] = [b,a];
console.log(`a = ${a} | b = ${b}`);  // a = 2 | b = 1

Use Case 3: Cloning arrays with the help of the spread operator.

const list = ['red', 'orange', 'yellow'];
const [...listClone] = list;

Use Case 4: Simplifying functions with parameter records.

A function parameter record allows using named arguments in faction calls instead of argument lists like so:

function displayName( paramRec) {
  alert( paramRec.first + " " + paramRec.last);
};
displayName({first:"James", last:"Bond"});

Using Destructuring, the parameter record fields are assigned to ordinary function parameters, simplifying the function's code:

function displayName({first, last}) {
  alert( first + " " + last);
}
displayName({first:"James", last:"Bond"});

Modules

... are (namespace-protected) libraries that explicitly export those (variable, function and class) names that other modules and web pages can use (as implicitly frozen/const). Modules need to explicitly import names from other modules, while web pages can load modules with a special script element.

For instance, suppose the following code is in a module file lib/math.js.

export function sum( x, y) {return x + y;};
export var pi = 3.141593;

Then, in another JS file, we can import all elements of lib/math.js and use them with

import * as math from "./lib/math.js";
console.log("2π = " + math.sum(math.pi, math.pi));

Or we can import specific elements of lib/math.js:

import {sum, pi} from "./lib/math.js";
console.log("2π = " + sum(pi, pi));

We can load modules in an HTML file in the following way:

<script type="module" async="async" src="module1.js"></script>
<script type="module" async="async" src="module2.js"></script>

Notice that there is no guarantee which will execute first due to asynchronous loading..

Important New JavaScript Elements for Advanced Programming

Promises for asynchronous method invocation
async-await for asynchronous method invocation

gwagner's blog

CRUD Operations with IndexedDB - Part 1: Creating and Clearing a Database

Category:

Asynchronous Programming in JavaScript

Callbacks

Promises

Calling asynchronous operations with `await`

Category:

The Most Important New JavaScript Elements for Basic Programming

Block-Scope Variable Declarations with `let` and `const`

Arrow Functions with `=>`

For-Of Loops over Iterable Objects

Template Literals

The Spread Operator `...`

Destructuring Assignments

Modules

Important New JavaScript Elements for Advanced Programming

gwagner's blog

CRUD Operations with IndexedDB - Part 1: Creating and Clearing a Database

Category:

Asynchronous Programming in JavaScript

Callbacks

Promises

Calling asynchronous operations with await

Category:

The Most Important New JavaScript Elements for Basic Programming

Block-Scope Variable Declarations with let and const

Arrow Functions with =>

For-Of Loops over Iterable Objects

Template Literals

The Spread Operator ...

Destructuring Assignments

Modules

Important New JavaScript Elements for Advanced Programming

Calling asynchronous operations with `await`

Block-Scope Variable Declarations with `let` and `const`

Arrow Functions with `=>`

The Spread Operator `...`