Question 60

Question

Explain how to use ReadableStream and WritableStream APIs for asynchronous data processing.

Answer

The ReadableStream and WritableStream APIs are designed for efficient asynchronous data handling, providing a more streamlined approach than callbacks or promises for tasks like reading and writing large files or streaming real-time data.

Core Concepts:

• Data Flow: Imagine a pipeline where data flows from a source (ReadableStream) through a series of transformations (optional) and finally to a destination (WritableStream).

• Asynchronous Nature: Operations on streams are non-blocking, meaning your code can continue executing other tasks while waiting for data to arrive or be processed.

Building Blocks:

1. ReadableStream: Represents a source of data. Examples include reading from a file system, network connection, or another stream.

2. WritableStream: Represents a destination for data. Examples include writing to a file system, network connection, or another stream.

3. TransformStream: A specialized type of stream that processes data as it flows through (e.g., filtering, encoding, encryption). It combines the capabilities of both ReadableStream and WritableStream.

Key Methods:

• readableStream.read() : Reads a chunk of data from the source stream.

• writableStream.write(chunk): Writes a chunk of data to the destination stream.

• transformStream.pipeTo(destination, options): Connects the transform stream to another ReadableStream or WritableStream.

Example: Reading and Writing Data Asynchronously

const fs = require('fs'); // Node.js file system module

// Read data from a file asynchronously using ReadableStream
const reader = fs.createReadStream('input.txt', { encoding: 'utf8' }); 

// Create a WritableStream to write the data to a new file
const writer = fs.createWriteStream('output.txt');

// Pipe the data from 'reader' to 'writer' (transforming it along the way)
reader
  .pipeTo(writer, { end: () => console.log('Data written successfully!') }); 

Benefits:

• Efficiency: Streams process data in chunks, minimizing memory usage for large datasets.

• Responsiveness: Asynchronous nature prevents your application from blocking while waiting for I/O operations to complete.

• Flexibility: TransformStream allows you to apply custom logic during data processing (e.g., compression, encryption).