< Back
calendar

Sep 22, 2019

What You Need to Know About Asynchronous Programming in JavaScript

You can’t afford to get this wrong.

Header for posting What You Need to Know About Asynchronous Programming in JavaScript
Photo by Toa Heftiba on Unsplash

Throughout my career I’ve been in a lot of job interviews, both as an interviewer and as a candidate. I’ve seen my share of candidates fall apart when asked to code asynchronous code using Promises and async/await.

Asynchronous programming is the bread and butter of JavaScript, yet so many developers fail to truly understand it for some reason.

Sure, they have worked with asynchronous code and they’ll recognise it when they see it. They know more or less how it works. But many fail to accurately reproduce it and understand all the essential implementation details.

Quite some developers I’ve interviewed are stuck on this level, which is actually sad.

If you want to be a serious JavaScript developer then asynchronous programming should be second nature. You should know this stuff like a baker knows bread.

Asynchronous programming can be tricky, but it’s not rocket science. There’s just some stuff that you need to know.

When you master these basics, you will have no trouble understanding the more advanced use cases and implementing them yourself.

What you already know

You already know how to work with Promise and async/await, at least I expect you to. If you don’t, check out these articles on MDN first and then come back.

What you should know

There are some patterns in asynchronous programming in JavaScript that keep coming back. These are very practical, all purpose solutions that you can and probably will apply very often and should be standard tools in your JavaScript toolbox.

Converting callback-based code into Promises

Callback-based code can be cumbersome to work with, especially when multiple chained calls and error handling are involved. This is basically the whole reason Promises exist, to make asynchronous programming easier.

It’s pretty straightforward to convert callback-based code into code that uses Promises.

Let take a look at the Node.js code to asynchronously read a file:

const fs = require('fs');

fs.readFile('/path/to/file', (err, file) => {
  if(err) {
    // handle the error
  }
  else {
    // do something with the file
  }
});

The fs.readFile method takes the path to a file and a callback as its arguments. When the file is read the callback is invoked with either an error as its first argument in case something went wrong, or null as its first argument and the file contents as the second argument in case of success.

It would be nice if we could use this method as a Promise like this:

fs.readFile('/path/to/file')
.then(file => {
  // do something with the file
})
.catch(err => {
  // handle the error
});

To accomplish this we can easily wrap it in a Promise:

const readFileAsync = path => {
  return new Promise((resolve, reject) => {
    fs.readFile(path, (err, file) => {
      return err ? reject(err) : resolve(file);
    });
  });
};

// usage
readFileAsync('/path/to/file')
.then(file => {
  // do something with the file
})
.catch(err => {
  // handle the error
});

The readFileAsync function takes a path as its argument and returns a new Promise. The Promise constructor takes a so-called executor function which in turn receives the resolve and reject callbacks.

These callbacks are what should be invoked in case of success and failure respectively. When the callback of the wrapped fs.readFile method receives an error it will be passed to reject. When it’s successful the received file will be passed to resolve.

If you’ve paid close attention you will have noticed that Promises are actually based on callbacks.

This is how you can turn any callback-based function into a Promise-based function.

Bonus tip: for Node.js you would probably use util.promisify.

You can do the same with event-based code. For example with FileReader. Let’s say you want to read a file in the browser and turn it into an ArrayBuffer:

const toArrayBuffer = blob => {
  const reader = new FileReader();

  reader.onload = e => {
    const buffer = e.target.result;
  };

  reader.onerror = e => {
    // handle the error
  };

  reader.readAsArrayBuffer(blob);
};

We can also turn this event-based code into Promises in the same way:

const toArrayBuffer = blob => {
  const reader = new FileReader();

  return new Promise((resolve, reject) => {
    reader.onload = e => resolve(e.target.result);

    reader.onerror = e => reject(e.target.error);

    reader.readAsArrayBuffer(blob);
  });
};

Here we basically did the same thing with events instead of a callback.

Using intermediate results in a Promise chain

If you work with Promises for a while you will run into the situation where you chain Promises and need to use intermediate results that are out of scope of a Promise callback:

const db = openDatabase();db.getUser(id)
.then(user => db.getOrders(user))
.then(orders => db.getProducts(orders[0]))
.then(products => {
  // cannot access orders here!
})

In this example we open a database connection and retrieve a user by its id, get the orders for that user and then the products that are in the user’s first order.

The problem here is that inside the callback for the Promise returned from db.getProducts() the orders variable is not accessible since it’s only defined in the scope of the callback for the previous Promise, returned from db.getOrders().

The simplest solution to this would be to initialise the orders variable in the outer scope so it will be available everywhere:

const db = openDatabase();

let orders;  // initialized in outer scope

db.getUser(id)
.then(user => db.getOrders(user))
.then(response => {
  orders = response;  // orders is assigned here

  return db.getProducts(orders[0]);
})
.then(products => {
  // orders is now accessible here!
})

It works, but it’s not the cleanest solution, especially when you have a complex Promise chain with many variables. This would result in a long list of variables that must be initialised.

Instead, you should use async and await since this is one of the main use cases. It gives you the power to use asynchronous code with synchronous syntax, so all variables share the same scope:

const db = openDatabase();

const getUserData = async id => {
  const user = await db.getUser(id);
  const orders = await db.getOrders(user);
  const products = await db.getProducts(orders[0]);

  return products;
};

getUserData(123);

Inside getUserData all variables now share the same scope and are all accessible in that scope, yet the code is fully asynchronous. The call to getUserData will not block any following code.

This is an example where async and await really shine.

You can combine async/await with Promises

How would you get the products returned from getUserData in the previous example?

Since getUserData is an async function you would use await inside another async function:

const getProducts = async () => {
  const products = await getUserData(123);
};

But you could also use .then:

getUserData(123)
.then(products => {
  // use products
})

This works because an async function always returns an implicit Promise.

If you’ve paid close attention you will have noticed that async/await is actually based on Promises.

What is good to know

As always, the devil is in the details and this is also true for asynchronous programming in JavaScript. I have often seen developers miss essential implementation details during job interviews, which demonstrates poor understanding of the concepts.

Promise callbacks always return a Promise

When chaining Promises you would typically return a Promise from every .then in the chain:

db.getUser(id)
.then(user => db.getOrders(user))
.then(orders => db.getProducts(orders[0]))
.then(products => db.getStats(products))

In the above example a Promise is returned from every callback inside .then, meaning that db.getOrders, db.getProducts and db.getStats all return a Promise.

But when you have a complex chain you will sooner or later need to return something that is not a Promise:

db.getUser(id)
.then(user => db.getOrders(user))
.then(orders => db.getProducts(orders[0]))
.then(products => products.length) // <-- oops, not a Promise!
.then(numberOfProducts => {
  db.saveStats(numberOfProducts);  return db.getStats(products);
})
...

However, the above code will run perfectly fine since any return value from a Promise callback inside .then or .catch will automatically be wrapped in a Promise.

This means you can return arbitrary values within a chain of Promises.

.then can actually take two arguments

Normally you would just pass one argument to .then, the callback that should be called when the Promise resolves. The callback that should be called when the Promise rejects is passes to .catch.

But .then can actually take both these callbacks, the first one for when the Promise resolves (success) and the second one for when the Promise rejects (error).

So instead of this:

fetch('http://some.domain.com')
.then(response => console.log('success'))
.catch(err => console.error('error'))

You could also do this:

fetch('http://some.domain.com')
.then(response => console.log('success'), 
      err => console.error('error'))

You could, but you shouldn't. Ever.

Update: some attentive readers on Reddit pointed out that there are actually use cases for this.

I was wrong.

For punishment I forced myself to write 67 levels deep callback hell code for a week straight.

That’ll teach me.

Read the updated text below:

The difference between .catch and the error callback as second argument to .then is that if an error occurs in a success callback anywhere in the Promise chain, it will be caught by .catch:

fetch('http://some.domain.com')
.then(response => response.json())
.then(json => fetch(...))
.then(response => response.text())
.then(text => ...)
.catch(err => console.error(err))  // any error will be caught here

If you pass both a success callback and an error callback to .then and an error occurs in the success callback, it won’t be caught by the error callback:

fetch('http://some.domain.com')
.then(successCallback,
      errorCallback) // an error in successCallback will NOT go here

It is very important to be aware of this and personally I always put a single .catch at the end of the chain, but there are actually use cases for an error handler as the second argument to .then.

Whenever you need to do fine-grained error handling or need to continue to handle the remaining promises in the chain after the point where the error occurred, this error handler as second argument is exactly what you need:

promise1()
.then(() => {
  // success callback
}, () => {
  // error callback, errors in promise1 will go here
  // if an error is thrown here or a rejected promise is returned
  // execution will go to the final error handler in .catch
  // if we return anything else, the chain will continue
})
.then(() => promise2())
.then(() => promise3())
.catch(finalErrorHandler);

In the above example, when an error occurs it will be caught by the error callback that is given as the second argument to .then. Inside that error handler we could decide what to do.

If we throw an error or return a rejected Promise, it will be caught by finalErrorHandler. If we return any other value, the chain will just continue and run promise2 and promise3. So in this case the chain will not be terminated but continue to run, effectively allowing for complex fine-grained error handling and recovery.

I haven’t come across such a scenario myself but this is a valid use case for the error handler as second argument. Just be aware that with this approach any errors occurring in success handlers won’t be caught.

What you may not know

Asynchronous programming can of course get much more complex, demanding more complex scenarios. I have asked candidates in job interviews to code a fairly simple scenario where an asynchronous API call has to be done conditionally.

Conditional asynchronous API call

Let’s say that an API call has to be done only when the user is logged in and then after that, some code has to be executed to delete the user’s session. But deleting the session can only be done after the API call is finished. If the user is not logged in, the API call is skipped and the session is deleted immediately.

When the user is logged in we can simply wait for the Promise returned from the API call to resolve and then delete the session inside the callback passed to .then:

if(userIsLoggedIn) {
  apiCall()
  .then(res => {
    deleteSession()
  })
}

If the user is not logged in we skip the API call and go straight to deleteSession. But since the API call is asynchronous, any code after it will be run immediately so we need to duplicate the call to deleteSession:

const checkLogin = () => {
  if(userIsLoggedIn) {
    apiCall()
    .then(res => {
      deleteSession()
    })
  }
  else {
    deleteSession();
  }
};

Now we have two calls to deleteSession which is not good. Since apiCall is asynchronous we have no choice but to run deleteSession inside .then and also create a Promise for when the user is not logged in:

const checkLogin = () => {
  if(userIsLoggedIn) {
    return apiCall();
  }
  else {
    return Promise.resolve(true);  // it works, but this is not good
  }
};checkLogin()
.then(() => {
  deleteSession();
})

The duplicate call to deleteSession is gone, but now we have to return a useless Promise from checkLogin when the user is not logged in, just so the code will execute in the right order.

This is the solution I’ve seen often in production code, but it’s definitely a code smell and should be avoided.

This example is still pretty simple, but when the code gets more complex this may turn into some hard to follow, error-prone code.

The only way to solve this in a clean and concise way is to use async and await:

const checkLogin = async () => {
  if(userIsLoggedIn) {
    await apiCall();
  }

  deleteSession();
};

Now when the user is logged in apiCall will be executed. Since checkLogin is now an async function and apiCall is prefixed with await, it will now wait until apiCall is finished and then run deleteSession.

When the user is not logged in, apiCall will simply be skipped and only deleteSession will be run. No more duplicate code, no more useless Promises.

That’s the power of async and await.

A Promise with a timeout

Another common scenario is when a timeout has to be set on a (possibly) long running API call.

Let’s say you call an API that has to respond within 5 seconds. If it doesn’t, a message should be returned to indicate the call took too long and has timed out.

I’ve seen developers struggle with this while getting entangled in multiple Promise spaghetti. The solution is actually quite simple to implement with Promise.race.

Promise.race takes an array (or other Iterable) of Promises and resolves or rejects as soon as one of the Promises in the array resolves or rejects. You can use this to pass the Promise you want to apply a timeout to together with another Promise that resolves/rejects after that time to Promise.race.

So if your Promise resolves within the specified timeout, all is fine. But if it doesn’t, the second Promise will resolve or reject after the timeout time and signal that there was a timeout:

const apiCall = url => fetch(url);const timeout = time => {
  return new Promise((resolve, reject) => {
    setTimeout(() => reject('Promise timed out'), time);
  });
};

Promise.race([apiCall(url), timeout(1000)])
.then(response => {
  // response from apiCall was successful 
})
.catch(err => {
  // handle the timeout
});

Beware though that any other error that may occur in apiCall will also go to the .catch handler, so you need to be able to determine if the error is the timeout or an actual error.

The key to understanding asynchronous programming in JavaScript

To truly understand asynchronous programming you need to understand the basics, the foundations it was built upon.

Remember:

async/await is based on Promises

Promises are based on callbacks

callbacks are the foundation of asynchronous programming in JavaScript

Too many developers I interviewed have only a vague or shallow understanding of how it more or less works, but this is not enough.

Only if you truly understand the foundation can you truly understand and ultimately, master asynchronous programming in JavaScript.

Join Modern Web Weekly, my weekly update on the modern web platform, web components, and Progressive Web Apps delivered straight to your inbox.