3.2 Default threading model
In the world of Javascript, there is one thing that is famous or rather infamous: single-threaded execution.
Javascript is well known for a single-threaded execution. There is a single poor thread that does everything. Everything literally means everything. Node.js made it known as the single-threaded event loop.
This single thread runs the JS code and also does everything that's required. Let's take an example and understand it better. Say there is a simple JS HTTP server that echoes whatever it receives from its clients. The poor single thread takes care of everything including:
- Execution of the JS code
- Listening to the socket for new TCP connections
- Processing all the concurrent incoming requests from the socket
- Creating high-level request and response objects
- JSON parsing
- Allocating a request-id (like a uuid)
- Encoding response into JSON
- Sending it over the TCP connection
- ....
For all the concurrent requests, all the above is done by a single thread. They fight for a piece of time from the single thread. Check the diagram shown above. All the requests run in the same thread. They fight to get scheduled.
In other languages like Java, there is generally a manager with a fixed size thread pool. The manager listens for incoming requests. On getting a new TCP connection, it takes a free thread out of the thread pool and hands over the request for further processing. For each incoming request, the manager finds a free thread and give it the request for further processing. If there are no free threads, the manager would queue the request till a thread gets freed up.
By reading the two ways i.e. single vs thread pool, an impression is formed that Javascript is very inefficient. It isn't that bad. It is of course slow, but it makes up in a clever way. In the JS world, execution happens on a single thread. This thread works on all the requests that come from all the clients. The only way a single thread can achieve all of this is because of asynchronicity and callbacks.
Deno also runs in a single-threaded environment. There is a single thread that does everything that has been shown in the previous section. However, if needed, there is a way to create additional application threads. These additional application threads are called web workers.
Web workers run independently. They aren't totally disconnected from their parent. Web workers can communicate to and fro with their parent thread through some kind of messaging. Except for the communication channel between the main thread and the web workers, all the workers are fully autonomous. For each web worker, a new V8 snapshot gets allocated. Each worker has its own event loop or tokio runtime.
Web workers are very useful in running CPU intensive operations that could choke the main thread. These special operations can be handed over to web workers so that the main thread could continue its processing.
Deno also follows the same principles and runs Javascript programs in a single loop. This would lead us to assume that the Deno process has a single thread. This is both right and wrong. Although Deno runs JS code in a single thread, Deno is a multithreaded process.
By default, all the extra threads come from the V8 engine. The V8 engine also runs the JS code in the context of the main thread. But to be able to run JS code very fast, V8 uses extra threads for CPU intensive work like garbage collection, ahead of time compilation, etc. It is possible for V8 to get configured to use a single thread, but that would negatively affect the performance as that single thread would need to do all the extra work too. That would reduce the availability of the main thread to run user programs. A single-threaded v8 could be disastrous.
When a Deno process is started with default options, it has 8 threads by default. This is the case when there are no web workers.
The distribution of threads is:
- There is one main worker thread or main thread
- There are seven V8 threads
= Total 8 threads
This is the main thread, owned and managed by Tokio Runtime. This is the thread that runs everything.
Here is the stack trace of the main thread:
2731 Thread_8461543 DispatchQueue_1: com.apple.main-thread (serial)
+ 2731 start (in libdyld.dylib) + 1 [0x7fff6dd53cc9]
+ 2731 main (in deno) + 418 [0x108ecfada]
+ 2731 std::sys_common::backtrace::__rust_begin_short_backtrace::h4e8d5235f9254db6 (in deno) + 10 [0x108d368d1]
+ 2731 deno::main::h7d1b5a97f8aef853 (in deno) + 10706 [0x108ec8676]
+ 2731 tokio::runtime::Runtime::block_on::h9f5c6c3dddbd431f (in deno) + 1768 [0x108d9fa5c]
+ 2731 _$LT$tokio..park..either..Either$LT$A$C$B$GT$$u20$as$u20$tokio..park..Park$GT$::park::h123a854f0de101c3 (in deno) + 204 [0x109439078]
+ 2731 _$LT$tokio..park..either..Either$LT$A$C$B$GT$$u20$as$u20$tokio..park..Park$GT$::park_timeout::h6834b5b57b34a394 (in deno) + 78 [0x109439308]
+ 2731 tokio::io::driver::Driver::turn::h1e669b6b05307d9f (in deno) + 72 [0x1094394df]
+ 2731 mio::poll::Poll::poll::hbd211acf9552cbd4 (in deno) + 763 [0x109148978]
+ 2731 kevent (in libsystem_kernel.dylib) + 10 [0x7fff6de99766]
As can be seen from the stack trace, the main thread is managed by the tokio runtime. This thread runs the tokio's event loop.
By default, there are seven v8 worker threads. These v8 worker threads do all the other work like garbage collection, runtime optimizations, JIT compilation, etc. The JS code doesn't run in these worker threads.
Here is the stack trace of one of the worker threads (all stack traces are the same):
2731 Thread_8461546: V8 DefaultWorke
+ 2731 thread_start (in libsystem_pthread.dylib) + 15 [0x7fff6df53b8b]
+ 2731 _pthread_start (in libsystem_pthread.dylib) + 148 [0x7fff6df58109]
+ 2731 v8::base::ThreadEntry(void*) (in deno) + 87 [0x10948c547]
+ 2731 v8::platform::DefaultWorkerThreadsTaskRunner::WorkerThread::Run() (in deno) + 31 [0x109491e6f]
+ 2731 v8::platform::DelayedTaskQueue::GetNext() (in deno) + 708 [0x109492644]
+ 2731 _pthread_cond_wait (in libsystem_pthread.dylib) + 698 [0x7fff6df58425]
+ 2731 __psynch_cvwait (in libsystem_kernel.dylib) + 10 [0x7fff6de97882]
If there is a web worker, the distribution of the threads is like this:
This just got interesting. With one web worker, there are a total of 9 threads. The distribution of threads is:
- There is one main worker thread or main thread
- There is one web worker thread
- There are seven V8 threads
= Total 9 threads
It's interesting to note that the number of v8 worker threads doesn't change with an additional web worker. The number of v8 worker threads can be configured for more, but it needs to come from the user. It all depends on the use case.
Stack trace of the new worker thread is:
2718 Thread_8525355: deno-worker-0
2718 thread_start (in libsystem_pthread.dylib) + 15 [0x7fff6df53b8b]
2718 _pthread_start (in libsystem_pthread.dylib) + 148 [0x7fff6df58109]
2718 std::sys::unix::thread::Thread::new::thread_start::hd4805e9612a32deb (in deno) + 45 [0x10a2e1b9d]
2718 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h7e30e5a55a4af9fa (in deno) + 116 [0x109d60e85]
2718 std::sys_common::backtrace::__rust_begin_short_backtrace::h30b57608d94f7821 (in deno) + 2856 [0x109d507e6]
2718 tokio::runtime::Runtime::block_on::h9f5c6c3dddbd431f (in deno) + 1768 [0x109dbaa5c]
2718 _$LT$tokio..park..either..Either$LT$A$C$B$GT$$u20$as$u20$tokio..park..Park$GT$::park::h123a854f0de101c3 (in deno) + 204 [0x10a454078]
2718 _$LT$tokio..park..either..Either$LT$A$C$B$GT$$u20$as$u20$tokio..park..Park$GT$::park_timeout::h6834b5b57b34a394 (in deno) + 78 [0x10a454308]
2718 tokio::io::driver::Driver::turn::h1e669b6b05307d9f (in deno) + 72 [0x10a4544df]
2718 mio::poll::Poll::poll::hbd211acf9552cbd4 (in deno) + 763 [0x10a163978]
2718 kevent (in libsystem_kernel.dylib) + 10 [0x7fff6de99766]
The above stack trace of the web worker is exactly the same as the stack trace from the main thread, except where the thread starts in Deno space. For the main thread, the thread started in deno::main, while for the web worker, the thread started in core::ops. Core's ops is the place where the web worker got spawned.
Let's see the last example. This is the distribution of 5 web worker threads:
The pattern is clearly visible. With one web worker, there are a total of 13 threads. The distribution of threads is:
- There is one main worker thread or main thread
- There are five web worker threads
- There are seven V8 threads
= Total 13 threads
We've just seen the threading model for Deno with and without web workers. Is that all the threads Deno would use? The answer is no. The ones that we've seen in this section were the OS aware static threads, meaning the ones that get created almost at the start. It is of course possible that applications can create web workers at runtime, but that'd be quite expensive.
There are more threads that could come in the future when the Deno process handles async ops. These dynamic threads or green threads are spawned and managed by tokio. These threads don't get scheduled by OS.