3.3 Asynchronous green threads

Overview

In Deno, the utilization of threads comes in a multifaceted manner, involving both OS-conscious static threads and OS-handled static threads, along with the integration of asynchronous green threads for executing asynchronous operations. It's essential to discern the nuances between static threads and green threads.

Static threads are assigned akin to regular threads and remain within the purview of the operating system's awareness. Consequently, the OS takes charge of their entire lifecycle, encompassing initiation and termination. The scheduling of these threads is orchestrated by the operating system, which ensures their orderly execution.

Conversely, asynchronous green threads operate distinctively as tokio tasks. The tokio runtime assumes the responsibility of managing the lifecycle of these threads. This signifies that the scheduling of green threads is orchestrated by the tokio runtime itself. Importantly, the operating system remains oblivious to the existence of tokio tasks or green threads. This unique characteristic lends a remarkable speed to tokio tasks, making them exceptionally efficient in operation. The synergy between Deno's threading mechanisms, including both static and green threads, enhances its capacity to seamlessly manage asynchronous tasks and deliver optimal performance.

Tokio tasks

Task or green thread

A task, also known as a green thread, represents a small and efficient unit of work that doesn't block the entire process. It's like a tiny worker that carries out a task without stopping everything else. Imagine it as a mini version of a computer task.

In the world of programming, tasks are a bit like OS threads, which are like separate paths of execution. However, they have a different way of being taken care of. While OS threads are looked after by the operating system's schedule, tasks get their attention from something called the Tokio runtime. This Tokio runtime is like a supervisor that makes sure each task gets its turn without waiting around.

Remember, green threads is just another way to talk about this kind of setup. It's a bit like a nickname. Now, let's peek at what makes these Tokio tasks special:

Lightweight

Tokio Tasks possess a quality of being lightweight. The Tokio runtime, rather than relying on the operating system, takes charge of scheduling tasks. This approach brings about a notable advantage: forming new tasks or shifting between existing ones doesn't necessitate a context switch, resulting in minimal additional processing. The cost associated with generating, executing, and concluding numerous tasks is kept admirably low, particularly when measured against conventional OS threads. As a result of these attributes, Tokio Tasks exhibit a remarkable efficiency in their performance.

Fast Scheduling

Tokio Tasks follow a cooperative scheduling approach. Unlike many operating systems that utilize preemptive multitasking, where threads are interrupted after a set time, Tokio Tasks embrace cooperative multitasking. In cooperative multitasking, a task runs until it explicitly relinquishes control, signifying to the Tokio runtime's scheduler that it's temporarily unable to proceed. This cooperative behavior allows a task to yield voluntarily, enabling the Tokio runtime to smoothly transition to executing the next task. Essentially, in the realm of Tokio, tasks are not forcibly paused until they willingly yield the control. This distinction in scheduling methodologies plays a pivotal role in Tokio's efficient and predictable concurrency management. By favoring cooperation over interruption, Tokio Tasks contribute to the overall stability and responsiveness of asynchronous applications.

Non-blocking

Tokio Tasks operate in a non-blocking manner. Normally, when an operating system thread engages in I/O activities or needs to coordinate with another thread, it becomes obstructed, allowing the operating system to arrange for the execution of a different thread. In situations where a task cannot proceed with its execution, it must instead relinquish control, enabling the Tokio runtime to arrange for the execution of a different task. It's generally advisable for tasks to refrain from carrying out system calls or other activities that could lead to the blocking of a thread. Doing so would impede the execution of other tasks that share the same thread. Keep in mind that these tasks don't correspond to actual threads; they're akin to virtual threads that operate within the scope of the main thread. Consequently, if a virtual thread encounters a blockage, it effectively halts the entire process. Rather than facing such a scenario, tasks should employ APIs designed to facilitate the execution of blocking operations within an asynchronous context. Tokio provides a range of equivalent asynchronous APIs that can be utilized for this purpose. This approach ensures the efficient execution of tasks and prevents bottlenecks that could arise from thread blocking.

Spawning tasks

Creating a tokio task or a green thread requires a process called spawning. This involves initiating the task or thread so that it can run concurrently with other tasks. There exist three distinct methods for spawning a tokio task, each serving its purpose in managing asynchronous operations.

task::spawn

The task::spawn function serves as an asynchronous counterpart to the thread::spawn function found in the standard library. Its purpose is to handle async blocks or other futures, initiating the creation of a fresh task to execute those tasks simultaneously.

By spawning a task, you enable it to operate concurrently alongside other tasks. This newly spawned task might execute on the ongoing thread or be directed to a separate thread for execution. The precise execution scenario hinges on the current configuration of the runtime environment.

It's important to note that there exists no assurance regarding the completion of a spawned task. In the event of a runtime shutdown, all ongoing tasks are abruptly terminated, regardless of their individual lifecycles.

use tokio::task;

task::spawn(async {
    // do async work
});

The Tokio spawn() function is utilized to execute a set of asynchronous instructions enclosed within an async block. This function provides back a "join handle," which essentially allows us to await the completion of the task that was spawned.

To better understand this concept, let's delve into a concrete example provided by the Tokio documentation:

use tokio::task;

#[tokio::main]
async fn main() {
    let join_handle = tokio::spawn(async {
        // Asynchronous operations to be performed here
        println!("Task has been spawned!");
    });

    // Waiting for the spawned task to complete
    join_handle.await.unwrap();
}

In this example, the tokio::spawn() function is employed to create a new asynchronous task. Inside the provided async block, various asynchronous operations can be executed. As soon as the task is spawned, the code continues running without waiting for the spawned task to finish.

To ensure that the main program waits for the spawned task to complete, the join_handle is utilized with the await keyword. This implies that the program will pause at this point and wait until the spawned task is done executing before continuing further.

In summary, the Tokio spawn() function facilitates the concurrent execution of asynchronous tasks, allowing us to efficiently manage and synchronize multiple asynchronous operations within our Rust programs.

task::spawn_blocking

This presents a unique method for crafting Tokio tasks that involve blocking the ongoing process. When a task within Tokio involves a process that blocks its execution, it can disrupt the entire flow, especially if that task shares the same thread with other tasks. In such cases, the entire thread can become stalled, preventing any other tasks from making progress.

To address this issue, there's a function known as task::spawn_blocking. This function works similarly to the task::spawn function we discussed earlier, but with a key difference. Instead of initiating a non-blocking future within the Tokio runtime, task::spawn_blocking initiates a blocking function on a distinct thread pool dedicated to handling such blocking tasks.

It's crucial to understand that employing a blocking call or engaging in extensive computations without yielding control is generally discouraged. Such practices can impede the executor's ability to drive other futures forward. However, with the approach provided by task::spawn_blocking, these concerns are mitigated. When you apply a closure using this method, it's executed on a separate thread pool designated for managing these kinds of blocking tasks. This arrangement ensures that the main futures executor remains unaffected and continues to operate smoothly. As a result, the execution of your program remains efficient and responsive.

use tokio::task;

task::spawn_blocking(|| {
    // do sync work
});

Blocking tasks are executed within a specialized thread pool designed to handle tasks that require synchronous operations or involve extensive computations. These tasks play a significant role in performing work that can't proceed concurrently, such as tasks that depend on previous steps or tasks that demand significant processing power.

Here is an example from tokio documentation:

use tokio::task;

let res = task::spawn_blocking(move || {
    // do some compute-heavy work or call synchronous code
    "done computing"
}).await?;

assert_eq!(res, "done computing");

task::block_in_place

The block_in_place function in Tokio serves a unique role in transitioning the current worker thread into a blocking thread. By doing so, it efficiently relocates other tasks that were initially executing on the same thread to a different worker thread. This strategic maneuver is employed to enhance performance by circumventing the need for frequent context switches.

What sets "block in place" apart is its distinctive ability to transform the current thread into a specialized blocking mode. Notably, Tokio's scheduler assumes the responsibility of orchestrating the migration of the remaining tasks away from this particular green thread. This delegation of task management contributes to the optimization of overall system operation.

use tokio::task;

task::block_in_place(|| {
    // do sync work
    
});

Deno's usage of tasks

The main purpose of Deno is to execute code in an asynchronous manner. Deno accomplishes this goal through two key methods:

  • Tokio's Asynchronous Functions: Deno includes a set of asynchronous functions from the Tokio library. These functions mirror the behavior of traditional synchronous functions that would normally block the program's flow.

  • Utilizing Tokio Tasks: In situations where Tokio doesn't offer predefined asynchronous functions, Deno employs tasks. These tasks enable Deno to execute functions asynchronously, even if they're not directly provided by the Tokio library.

This asynchronous execution is crucial because it allows Deno to efficiently manage multiple tasks simultaneously, enhancing the overall performance and responsiveness of applications. By leveraging Tokio's asynchronous capabilities and employing tasks when necessary, Deno provides a versatile environment for developing applications that can handle complex and concurrent tasks without sacrificing efficiency.

Deno operates in a distinct manner compared to task::spawn, which generates a fresh task. This choice is possibly motivated by the pursuit of enhanced performance. Whenever a basic spawn is executed, a novel green thread is brought forth, which might result in a small extra load when carrying out uncomplicated asynchronous code. In lieu of the conventional spawn approach, Deno employs a strategy called spawn_blocking to manage asynchronous operations. However, this method is predominantly employed for tasks associated with the file system. For alternative asynchronous activities, Deno directly capitalizes on the available asynchronous APIs, which are attainable either through tokio or other external libraries. To reinforce our understanding, let's delve into an illustrative example of each of these methodologies as employed within the Deno environment.

Async op without a task

To begin, let's delve into an illustrative instance of an asynchronous operation that avoids the necessity of explicitly generating a task. Deno operates in a manner that obviates the requirement for spawning tasks, owing to its seamless integration of async functions. A case in point pertains to the process of handling an asynchronous TCP connection. Ordinarily, this action would be accompanied by a blocking procedure. However, Deno employs a low-level operation, aptly named "op_accept," to execute the reception of TCP connections in an asynchronous fashion. This distinctive operation adeptly embraces incoming TCP connections.

In further detail, Deno's approach to TCP connection acceptance involves the utilization of the "op_accept" operation. Traditionally, this endeavor would necessitate intricate handling to prevent blocking, but Deno's innovative design sidesteps such complexities. In the realm of asynchronous programming, other frameworks like tokio have already incorporated support for asynchronous TCP and UDP. This convergence means that Deno can seamlessly leverage the existing asynchronous infrastructure without necessitating any specialized interventions.

async fn op_accept(
  state: Rc<RefCell<OpState>>,
  rid: ResourceId,
) -> Result<ResourceId, Error> {
  let listener = state.borrow().resource_table.get::<TcpListener>(rid)?;
  let stream = listener.accept().await?;
  let rid = state.borrow_mut().resource_table.add(stream);
  Ok(rid)
}

// TcpListener

struct TcpListener {
  inner: tokio::net::TcpListener,
}

As previously mentioned, Deno makes direct use of asynchronous functions provided by Tokio.

Async op with blocking task

Next, let's delve into an illustrative instance where Deno transforms synchronous operations into asynchronous tasks. A prime example of this conversion involves file system operations. In the realm of Deno, exclusively asynchronous file system operations are initiated with a blocking nature.

Let's explore a few of these asynchronous file system operations:

  • Opening files

  • Seeking within files

  • Gathering file status information (Fstat)

  • Creating directories (Mkdir)

  • Adjusting file permissions (Chmod)

  • Modifying file ownership (Chown)

  • Copying files

  • Reading directories

This list, though not exhaustive due to its considerable length, showcases the breadth of asynchronous file system operations. It's important to emphasize that only the asynchronous file system operations are executed as blocking tasks. This entails that these operations are likely to be orchestrated on Tokio's dedicated thread pool, enhancing their efficiency.

Taking this further, it's worth understanding the synchronous and asynchronous implementation of a fundamental operation like 'opening a file':

fn open_sync(
    &self,
    path: &Path,
    options: OpenOptions,
  ) -> FsResult<Rc<dyn File>> {
    let opts = open_options(options);
    let std_file = opts.open(path)?;
    Ok(Rc::new(StdFileResourceInner::file(std_file)))
  }
  
  async fn open_async(
    &self,
    path: PathBuf,
    options: OpenOptions,
  ) -> FsResult<Rc<dyn File>> {
    let opts = open_options(options);
    let std_file = spawn_blocking(move || opts.open(path)).await??;
    Ok(Rc::new(StdFileResourceInner::file(std_file)))
  }

These two functions share many similarities. They both invoke the same opts.open(path); function. Yet, a significant distinction exists between them.

In the synchronous version, when the call to set the file time is made, it obstructs the ongoing thread until opts.open(path) completes its task.

In contrast, the asynchronous version initiates a blocking task that handles the call to set the file time. This approach avoids blocking the thread directly, as the task operates within a dedicated thread pool managed by tokio. At the conclusion of the spawn_blocking operation, an await instruction is employed. This await patiently waits for the task to finalize its execution before moving forward with subsequent actions.

// Sync

let std_file = opts.open(path)?;

// Async

let std_file = spawn_blocking(move || opts.open(path)).await??;

--

This section covered the asynchronous abilities provided by Tokio using its built-in APIs or tasks. This involves understanding how Tokio helps handle tasks that can run independently without blocking other processes.

Last updated