Parallel execution of operations — JS: Asynchronous programming

Let's try to make several asynchronous calls at the same time and then make use of the result. Let's try to rewrite our file merging task. Here's the text:

Suppose we've been set the task of reading the contents of two files and writing them to a third (merging files)

From this statement, you can see that both source files can be read simultaneously, and once they've both been read, they can be written to a new file.

fs.readFile('./first', 'utf-8', (error1, data1) => {
  // ?
});

fs.readFile('./second', 'utf-8', (error2, data2) => {
  // ?
});

Since our code is asynchronous, you can only get the result of each function inside the callbacks. Moreover, we can't know the order the callbacks run in - it all depends on which file is read faster. To track the execution status of these operations, we have to introduce a global state (relative to these operations), which we'll use to track if the task is complete, and which we'll use to save data in. And only when all operations are finished will we write the old files to the new one. In addition, we need to clearly separate the data from the first and second files, because writing to a new file, unlike reading, must take place in a certain order.

const state = {
  count: 0,
  results: [],
};

fs.readFile('./first', 'utf-8', (error1, data1) => {
  state.count += 1;
  state.results[0] = data1;
});

fs.readFile('./second', 'utf-8', (error2, data2) => {
  state.count += 1;
  state.results[1] = data2;
});

When both operations are complete, the state will be filled with data and count will become 2. It's this condition that we'll tie our code to:

import fs from 'fs';

const state = {
  count: 0,
  results: [],
};

const tryWriteNewFile = () => {
  if (state.count !== 2) {
    return; // guard expression
  }

  fs.writeFile('./new-file', state.results.join(''), (error) => {
    if (error) {
      return;
    }
    console.log('finished!');
  });
};

console.log('first reading was started');
fs.readFile('./first', 'utf-8', (error1, data1) => {
  console.log('first callback');
  if (error1) {
    return;
  }
  state.count += 1;
  state.results[0] = data1;
  tryWriteNewFile();
});

console.log('second reading was started');
fs.readFile('./second', 'utf-8', (error2, data2) => {
  console.log('second callback');
  if (error2) {
    return;
  }
  state.count += 1;
  state.results[1] = data2;
  tryWriteNewFile();
});

// One run
// node index.js
// first reading was started
// second reading was started
// second callback
// first callback
// finished!

// Another run
// node index.js
// first reading was started
// second reading was started
// first callback
// second callback
// finished!

Now the files are read in parallel, and we've finally seen the advantage of simultaneous asynchronous operations in practice. This program is executed much faster than the synchronous version! Moreover, the larger the size of the files, the greater the difference. However, it's worth noting that although the reading of the files is done in parallel, the work done by js itself, which processes the result, is strictly sequential. Callbacks only start to run after the current call stack is empty and in the order in which the asynchronous operations are done (parallel running does not mean that operations start and end at the same time).

Writing this sort of code every time is very tedious, so it's better to use a library called async, which provides a set of ready-made abstractions to work in asynchronous code. It contains dozens of functions for a large number of tasks related to the ordering of asynchronous operations. Below is an example of a solution to our problem using this library:

import { map } from 'async';
import fs from 'fs';

map(['./first', './second'], fs.readFile, (err1, results) => {
  if (err1) {
    return;
  }
  fs.writeFile('./new-file', results.join(''), (err2) => {
    if (err2) {
      return;
    }
    console.log('finished!');
  });
});

You have to agree that this is much better ;) But, as you'll see later, you can go even further.

Recommended materials

1. async - library for working in asynchronous style

Are there any more questions? Ask them in the Discussion section.

The Hexlet support team or other students will answer you.