3.3 Asynchronous green threads

Overview

Deno uses threads in multiple ways: OS-aware static threads, OS-handled static threads, and asynchronous green threads for executing asynchronous operations. Understanding the differences between static threads and green threads is crucial.

Static threads are like regular threads and are managed by the operating system. The OS controls their entire lifecycle, from creation to termination, and schedules them for execution. This means that the operating system is responsible for initiating and terminating static threads, and it also handles their scheduling.

Asynchronous green threads, on the other hand, are managed by the tokio runtime as tokio tasks. The tokio runtime schedules these threads, and the operating system is not aware of their existence. This makes tokio tasks very efficient, as they are not subject to the overhead of the operating system's scheduling mechanisms. The combination of Deno's threading mechanisms, including static and green threads, allows it to manage asynchronous tasks efficiently and deliver optimal performance.

Tokio tasks

Task or green thread

A task, also known as a green thread, represents a small and efficient unit of work that does not block the entire process. It is similar to a tiny worker that carries out a task without stopping everything else. Imagine it as a mini version of a computer task.

In the world of programming, tasks are similar to OS threads, which are separate paths of execution. However, they are managed differently. While OS threads are scheduled by the operating system's scheduler, tasks are managed by the Tokio runtime. The Tokio runtime acts as a supervisor, ensuring each task gets its turn without waiting. This means that tasks are executed efficiently and without blocking the entire process.

The term "green threads" is a nickname for this setup. It is important to understand that green threads are not actual threads but rather a way to manage tasks efficiently. Now, let's explore what makes Tokio tasks special:

Lightweight

Tokio Tasks have the characteristic of being lightweight. The Tokio runtime is responsible for scheduling tasks, rather than relying on the operating system. This approach offers a significant advantage: creating new tasks or switching between existing ones does not require a context switch, resulting in minimal additional processing overhead. The cost of creating, executing, and completing multiple tasks is remarkably low, especially when compared to traditional OS threads. As a result, Tokio Tasks demonstrate exceptional efficiency in their performance.

Fast Scheduling

Tokio Tasks use a cooperative scheduling approach, unlike many operating systems that use preemptive multitasking. In preemptive multitasking, threads are interrupted after a set time. In contrast, Tokio Tasks use cooperative multitasking, where a task runs until it voluntarily gives up control. This allows the Tokio runtime's scheduler to smoothly switch to the next task. In Tokio, tasks are not forced to pause until they choose to yield control. This difference in scheduling methods is crucial for Tokio's efficient and predictable concurrency management. By prioritizing cooperation over interruption, Tokio Tasks enhance the stability and responsiveness of asynchronous applications.

Non-blocking

Tokio Tasks operate in a non-blocking manner. Normally, when an operating system thread engages in I/O activities or needs to coordinate with another thread, it becomes obstructed, allowing the operating system to arrange for the execution of a different thread. In situations where a task cannot proceed with its execution, it must instead relinquish control, enabling the Tokio runtime to arrange for the execution of a different task. It's generally advisable for tasks to refrain from carrying out system calls or other activities that could lead to the blocking of a thread. Doing so would impede the execution of other tasks that share the same thread. Keep in mind that these tasks don't correspond to actual threads; they're akin to virtual threads that operate within the scope of the main thread.

Consequently, if a virtual thread encounters a blockage, it effectively halts the entire process. Rather than facing such a scenario, tasks should employ APIs designed to facilitate the execution of blocking operations within an asynchronous context. Tokio provides a range of equivalent asynchronous APIs that can be utilized for this purpose. This approach ensures the efficient execution of tasks and prevents bottlenecks that could arise from thread blocking.

Spawning tasks

Creating a tokio task or a green thread requires a process called spawning. This involves initiating the task or thread so that it can run concurrently with other tasks. There exist three distinct methods for spawning a tokio task, each serving its purpose in managing asynchronous operations.

task::spawn

The task::spawn function serves as an asynchronous counterpart to the thread::spawn function found in the standard library. Its purpose is to handle async blocks or other futures, initiating the creation of a fresh task to execute those tasks simultaneously.

By spawning a task, you enable it to operate concurrently alongside other tasks. This newly spawned task might execute on the ongoing thread or be directed to a separate thread for execution. The precise execution scenario hinges on the current configuration of the runtime environment.

It's important to note that there exists no assurance regarding the completion of a spawned task. In the event of a runtime shutdown, all ongoing tasks are abruptly terminated, regardless of their individual lifecycles.

use tokio::task;

task::spawn(async {
    // do async work
});

The Tokio spawn() function is utilized to execute a set of asynchronous instructions enclosed within an async block. This function provides back a "join handle," which essentially allows us to await the completion of the task that was spawned.

To better understand this concept, let's check a concrete example provided by the Tokio documentation:

use tokio::task;

#[tokio::main]
async fn main() {
    let join_handle = tokio::spawn(async {
        // Asynchronous operations to be performed here
        println!("Task has been spawned!");
    });

    // Waiting for the spawned task to complete
    join_handle.await.unwrap();
}

In this example, the tokio::spawn() function is used to create a new asynchronous task. Inside the provided async block, various asynchronous operations can be executed. As soon as the task is spawned, the code continues running without waiting for the spawned task to finish.

To ensure that the main program waits for the spawned task to complete, the join_handle is used with the await keyword. This implies that the program will pause at this point and wait until the spawned task is done executing before continuing further.

In summary, the Tokio spawn() function facilitates the concurrent execution of asynchronous tasks, allowing us to efficiently manage and synchronize multiple asynchronous operations within our Rust programs.

task::spawn_blocking

This presents a unique method for creating Tokio tasks that involve blocking the ongoing process. When a task within Tokio involves a process that blocks its execution, it can disrupt the entire flow, especially if that task shares the same thread with other tasks. In such cases, the entire thread can become stalled, preventing any other tasks from making progress.

To address this issue, there's a function known as task::spawn_blocking. This function works similarly to the task::spawn function we discussed earlier, but with a key difference. Instead of initiating a non-blocking future within the Tokio runtime, task::spawn_blocking initiates a blocking function on a distinct thread pool dedicated to handling such blocking tasks.

It's crucial to understand that using a blocking call or engaging in extensive computations without yielding control is generally discouraged. Such practices can impede the executor's ability to drive other futures forward. However, with the approach provided by task::spawn_blocking, these concerns are mitigated. When you apply a closure using this method, it's executed on a separate thread pool designated for managing these kinds of blocking tasks. This arrangement ensures that the main futures executor remains unaffected and continues to operate smoothly. As a result, the execution of your program remains efficient and responsive.

use tokio::task;

task::spawn_blocking(|| {
    // do sync work
});

Blocking tasks are executed within a specialized thread pool designed to handle tasks that require synchronous operations or involve extensive computations. These tasks play a significant role in performing work that can't proceed concurrently, such as tasks that depend on previous steps or tasks that demand significant processing power.

Here is an example from tokio documentation:

use tokio::task;

let res = task::spawn_blocking(move || {
    // do some compute-heavy work or call synchronous code
    "done computing"
}).await?;

assert_eq!(res, "done computing");

task::block_in_place

The block_in_place function in Tokio plays a unique role in converting the current worker thread into a blocking thread. This conversion efficiently relocates other tasks executing on the same thread to a different worker thread, enhancing performance by reducing context switches.

Unlike other functions, block_in_place has the distinctive ability to transform the current thread into a specialized blocking mode. Tokio's scheduler manages the migration of remaining tasks away from this green thread, optimizing system operation.

use tokio::task;

task::block_in_place(|| {
    // do sync work
    
});

Deno's usage of tasks

The main purpose of Deno is to execute code in an asynchronous manner. Deno accomplishes this goal through two key methods:

• Tokio's Asynchronous Functions: Deno includes a set of asynchronous functions from the Tokio library. These functions mirror the behavior of traditional synchronous functions that would normally block the program's flow.

• Utilizing Tokio Tasks: In situations where Tokio doesn't offer predefined asynchronous functions, Deno uses tasks. These tasks enable Deno to execute functions asynchronously, even if they're not directly provided by the Tokio library.

This asynchronous execution is crucial because it allows Deno to efficiently manage multiple tasks simultaneously, enhancing the overall performance and responsiveness of applications. By leveraging Tokio's asynchronous capabilities and using tasks when necessary, Deno provides a versatile environment for developing applications that can handle complex and concurrent tasks without sacrificing efficiency.

Deno operates in a distinct manner compared to task::spawn, which generates a fresh task. This choice is possibly motivated by the pursuit of enhanced performance. Whenever a basic spawn is executed, a novel green thread is brought forth, which might result in a small extra load when carrying out uncomplicated asynchronous code. In lieu of the conventional spawn approach, Deno uses a strategy called spawn_blocking to manage asynchronous operations. However, this method is predominantly used for tasks associated with the file system. For alternative asynchronous activities, Deno directly capitalizes on the available asynchronous APIs, which are attainable either through tokio or other external libraries. To reinforce our understanding, let's get into an illustrative example of each of these methodologies as employed within the Deno environment.

Async op without a task

To start, let's look at an example of an asynchronous operation that doesn't need to create a task explicitly. Deno handles async functions seamlessly, making task creation unnecessary. For instance, handling an asynchronous TCP connection typically involves a blocking procedure. However, Deno uses a low-level operation called "op_accept" to receive TCP connections asynchronously. This operation efficiently handles incoming TCP connections.

Deno's approach to accepting TCP connections uses the "op_accept" operation. Normally, this would require complex handling to avoid blocking, but Deno's design avoids these complexities. Other frameworks like Tokio already support asynchronous TCP and UDP, making it easy for Deno to use the existing infrastructure without needing special interventions. This approach makes Deno's asynchronous programming more efficient and scalable. Deno's innovative design and seamless integration of async functions make it a popular choice for developers.

async fn op_accept(
  state: Rc<RefCell<OpState>>,
  rid: ResourceId,
) -> Result<ResourceId, Error> {
  let listener = state.borrow().resource_table.get::<TcpListener>(rid)?;
  let stream = listener.accept().await?;
  let rid = state.borrow_mut().resource_table.add(stream);
  Ok(rid)
}

// TcpListener

struct TcpListener {
  inner: tokio::net::TcpListener,
}

As previously mentioned, Deno makes direct use of asynchronous functions provided by Tokio.

Async op with blocking task

Next, let's look at an example where Deno converts synchronous operations to asynchronous tasks. A good example is file system operations. In Deno, all file system operations are asynchronous and start with a blocking call.

Here are some examples of asynchronous file system operations:

• Opening files

• Seeking within files

• Getting file status information (Fstat)

• Creating directories (Mkdir)

• Changing file permissions (Chmod)

• Changing file ownership (Chown)

• Copying files

• Reading directories

This list shows the range of asynchronous file system operations. It's important to note that only asynchronous file system operations are executed as blocking tasks. This means they are likely to be handled by Tokio's dedicated thread pool, making them more efficient.

To illustrate further, let's compare the synchronous and asynchronous implementation of a basic operation like 'opening a file':

fn open_sync(
    &self,
    path: &Path,
    options: OpenOptions,
  ) -> FsResult<Rc<dyn File>> {
    let opts = open_options(options);
    let std_file = opts.open(path)?;
    Ok(Rc::new(StdFileResourceInner::file(std_file)))
  }
  
  async fn open_async(
    &self,
    path: PathBuf,
    options: OpenOptions,
  ) -> FsResult<Rc<dyn File>> {
    let opts = open_options(options);
    let std_file = spawn_blocking(move || opts.open(path)).await??;
    Ok(Rc::new(StdFileResourceInner::file(std_file)))
  }

These two functions have many similarities. They both call the same function, opts.open(path). However, there is a significant difference between them.

In the synchronous version, the call to set the file time blocks the current thread until opts.open(path) finishes its task.

In contrast, the asynchronous version starts a blocking task that handles the call to set the file time. This approach avoids blocking the thread directly, as the task runs in a dedicated thread pool managed by Tokio. After starting the task with spawn_blocking, the code uses the await instruction to wait for the task to finish before continuing with the next steps.

// Sync

let std_file = opts.open(path)?;

// Async

let std_file = spawn_blocking(move || opts.open(path)).await??;

--

This section covered the asynchronous abilities provided by Tokio using its built-in APIs or tasks. This involves understanding how Tokio helps handle tasks that can run independently without blocking other processes.