6.6 Registration of ops
Ops are an integral part of Deno. Ops are the most common entities that cross the bridge between JS and Rust. This is where the boundary is crossed back and forth. In this section, we'll go over how ops get registered. In a later section, we'll see how ops are triggered at runtime and how the responses are handled.
Ops are low-level functions that are implemented in Rust to support the high-level functions in JS. For example,
Deno.getEnv() is a high-level function, and the corresponding Rust op is implemented by op_get_env.// HIGH-LEVEL JS FUNCTION
function getEnv(key) {
return core.jsonOpSync("op_get_env", { key })[0];
}
// LOW-LEVEL RUST FUNCTION
fn op_get_env(
state: &mut OpState,
args: Value,
_zero_copy: &mut [ZeroCopyBuf],
) -> Result<Value, AnyError> {
let args: GetEnv = serde_json::from_value(args)?;
state.borrow::<Permissions>().check_env()?;
let r = match env::var(args.key) {
Err(env::VarError::NotPresent) => json!([]),
v => json!([v?]),
};
Ok(r)
}
All the ops follow the same pattern, from initialization to processing the result. Following are the typical phases of an op:
Deno ops go through a number of stages. The above diagram shows the typical op state machine. Let's go through the states one by one:
- Register handler
- Registers op handler
- Waiting to get called
- This is like an idle state as the op is waiting to get called
- Dispatch
- Dispatch means sending crossing the bridge from v8 to Deno
- Execute
- Deno runs the op handler
- Process result
- The result is sent back to v8 where it gets processed in JS
- This is like coming back from the bridge.
The process of dispatching and processing result happens at runtime. We'll see it in detail later. For now, we'll focus on op registration.
All the ops fall into one of the two categories:
- Sync op
- Request and response are processed synchronously
- Execution is blocked
- Async op
- Request and response are processed asynchronously
- Execution is not blocked
Some of the ops fall in either of these categories while some fall in both. For example, get and set env are sync ops. Another example is reading a file that has both sync and async ops. It depends on the op and the user. For some of the ops, the async mode doesn't make sense. This type of ops has only sync variants. While for some of the ops like reading a file, the decision is with the user. If the file is very big, the user may choose to go for async reading.
Op registration happens at startup. Deno registers all of its low-level ops right at the process startup. Most of the op registration happens in the main worker initialization. Some of the ops get registered at other places, but that number is very less.
Here is the main worker initialization code for a quick recall:
// MainWorker
pub fn from_options(
main_module: ModuleSpecifier,
permissions: Permissions,
options: &WorkerOptions,
) -> Self {
// CODE OMITTED ---
let js_runtime = &mut worker.js_runtime;
{
// All ops registered in this function depend on these
{
let op_state = js_runtime.op_state();
let mut op_state = op_state.borrow_mut();
op_state.put::<Metrics>(Default::default());
op_state.put::<Permissions>(permissions);
op_state.put::<ops::UnstableChecker>(ops::UnstableChecker {
unstable: options.unstable,
});
}
ops::runtime::init(js_runtime, main_module);
ops::fetch::init(
js_runtime,
options.user_agent.clone(),
options.ca_filepath.as_deref(),
);
ops::timers::init(js_runtime);
ops::worker_host::init(
js_runtime,
None,
options.create_web_worker_cb.clone(),
);
ops::crypto::init(js_runtime, options.seed);
ops::reg_json_sync(js_runtime, "op_close", deno_core::op_close);
ops::reg_json_sync(js_runtime, "op_resources", deno_core::op_resources);
ops::reg_json_sync(
js_runtime,
"op_domain_to_ascii",
deno_web::op_domain_to_ascii,
);
ops::fs_events::init(js_runtime);
ops::fs::init(js_runtime);
ops::io::init(js_runtime);
ops::net::init(js_runtime);
ops::os::init(js_runtime);
ops::permissions::init(js_runtime);
ops::plugin::init(js_runtime);
ops::process::init(js_runtime);
ops::signal::init(js_runtime);
ops::tls::init(js_runtime);
ops::tty::init(js_runtime);
ops::websocket::init(
js_runtime,
options.ca_filepath.as_deref(),
options.user_agent.clone(),
);
}
// -- CODE OMITTED ---
}
Let's look at a couple of the init functions:
// crypto::init
pub fn init(rt: &mut deno_core::JsRuntime, maybe_seed: Option<u64>) {
if let Some(seed) = maybe_seed {
let rng = StdRng::seed_from_u64(seed);
let op_state = rt.op_state();
let mut state = op_state.borrow_mut();
state.put::<StdRng>(rng);
}
super::reg_json_sync(rt, "op_get_random_values", op_get_random_values);
}
// net::init
pub fn init(rt: &mut deno_core::JsRuntime) {
super::reg_json_async(rt, "op_accept", op_accept);
super::reg_json_async(rt, "op_connect", op_connect);
super::reg_json_async(rt, "op_shutdown", op_shutdown);
super::reg_json_sync(rt, "op_listen", op_listen);
super::reg_json_async(rt, "op_datagram_receive", op_datagram_receive);
super::reg_json_async(rt, "op_datagram_send", op_datagram_send);
}
At the time of registration, an op gets registered as an async or a sync op. This would get used when the op is called at runtime.
As there are two types of ops, there are two ways to register ops:
- reg_json_sync
- reg_json_async
pub fn reg_json_async<F, R>(rt: &mut JsRuntime, name: &'static str, op_fn: F)
where
F: Fn(Rc<RefCell<OpState>>, Value, BufVec) -> R + 'static,
R: Future<Output = Result<Value, AnyError>> + 'static,
{
rt.register_op(name, metrics_op(json_op_async(op_fn)));
}
pub fn reg_json_sync<F>(rt: &mut JsRuntime, name: &'static str, op_fn: F)
where
F: Fn(&mut OpState, Value, &mut [ZeroCopyBuf]) -> Result<Value, AnyError>
+ 'static,
{
rt.register_op(name, metrics_op(json_op_sync(op_fn)));
}
Depending on the type of op, the op function like
op_get_random_values or op_datagram_send gets wrapped into a core function like json_op_sync and json_op_async. Wrapping ops into core functions ensures that, for all ops, requests and responses follow the same protocol.Leaving the core json_op functions aside for time being, all the ops get added to an op table:
pub fn register_op<F>(&mut self, name: &str, op_fn: F) -> OpId
where
F: Fn(Rc<RefCell<OpState>>, BufVec) -> Op + 'static,
{
let (op_id, prev) = self.0.insert_full(name.to_owned(), Rc::new(op_fn));
assert!(prev.is_none());
op_id
}
All the ops are embedded into either json_op_sync or json_op_async. These utility functions act as an interface layer between JS and Rust. These functions ensure that all the ops process requests and return responses in the same format.
This function makes a call to the op and returns a response immediately.
pub fn json_op_sync<F>(op_fn: F) -> Box<OpFn>
where
F: Fn(&mut OpState, Value, &mut [ZeroCopyBuf]) -> Result<Value, AnyError>
+ 'static,
{
debug!("json_op_async");
Box::new(move |state: Rc<RefCell<OpState>>, mut bufs: BufVec| -> Op {
let result = serde_json::from_slice(&bufs[0])
.map_err(AnyError::from)
.and_then(|args| op_fn(&mut state.borrow_mut(), args, &mut bufs[1..]));
let buf =
json_serialize_op_result(None, result, state.borrow().get_error_class_fn);
Op::Sync(buf)
})
}
This function gets the promise id from the args, creates a future to call the op, and returns the future to the caller. We'll go over the use of the promise id when we discuss async ops.
pub fn json_op_async<F, R>(op_fn: F) -> Box<OpFn>
where
F: Fn(Rc<RefCell<OpState>>, Value, BufVec) -> R + 'static,
R: Future<Output = Result<Value, AnyError>> + 'static,
{
let try_dispatch_op =
move |state: Rc<RefCell<OpState>>, bufs: BufVec| -> Result<Op, AnyError> {
let args: Value = serde_json::from_slice(&bufs[0])?;
let promise_id = args
.get("promiseId")
.and_then(Value::as_u64)
.ok_or_else(|| type_error("missing or invalid `promiseId`"))?;
let bufs = bufs[1..].into();
use crate::futures::FutureExt;
let fut = op_fn(state.clone(), args, bufs).map(move |result| {
json_serialize_op_result(
Some(promise_id),
result,
state.borrow().get_error_class_fn,
)
});
Ok(Op::Async(Box::pin(fut)))
};
Box::new(move |state: Rc<RefCell<OpState>>, bufs: BufVec| -> Op {
match try_dispatch_op(state.clone(), bufs) {
Ok(op) => op,
Err(err) => Op::Sync(json_serialize_op_result(
None,
Err(err),
state.borrow().get_error_class_fn,
)),
}
})
}
At the time of writing, there are 115 ops that get registered. Here is the complete list:
1 op_main_module
2 op_metrics
3 op_fetch
4 op_fetch_read
5 op_create_http_client
6 op_global_timer_stop
7 op_global_timer_start
8 op_global_timer
9 op_now
10 op_sleep_sync
11 op_create_worker
12 op_host_terminate_worker
13 op_host_post_message
14 op_host_get_message
15 op_host_unhandled_error
16 op_get_random_values
17 op_close
18 op_resources
19 op_domain_to_ascii
20 op_fs_events_open
21 op_fs_events_poll
22 op_open_sync
23 op_open_async
24 op_seek_sync
25 op_seek_async
26 op_fdatasync_sync
27 op_fdatasync_async
28 op_fsync_sync
29 op_fsync_async
30 op_fstat_sync
31 op_fstat_async
32 op_umask
33 op_chdir
34 op_mkdir_sync
35 op_mkdir_async
36 op_chmod_sync
37 op_chmod_async
38 op_chown_sync
39 op_chown_async
40 op_remove_sync
41 op_remove_async
42 op_copy_file_sync
43 op_copy_file_async
44 op_stat_sync
45 op_stat_async
46 op_realpath_sync
47 op_realpath_async
48 op_read_dir_sync
49 op_read_dir_async
50 op_rename_sync
51 op_rename_async
52 op_link_sync
53 op_link_async
54 op_symlink_sync
55 op_symlink_async
56 op_read_link_sync
57 op_read_link_async
58 op_ftruncate_sync
59 op_ftruncate_async
60 op_truncate_sync
61 op_truncate_async
62 op_make_temp_dir_sync
63 op_make_temp_dir_async
64 op_make_temp_file_sync
65 op_make_temp_file_async
66 op_cwd
67 op_futime_sync
68 op_futime_async
69 op_utime_sync
70 op_utime_async
71 op_read
72 op_write
73 op_accept
74 op_connect
75 op_shutdown
76 op_listen
77 op_datagram_receive
78 op_datagram_send
79 op_exit
80 op_env
81 op_exec_path
82 op_set_env
83 op_get_env
84 op_delete_env
85 op_hostname
86 op_loadavg
87 op_os_release
88 op_system_memory_info
89 op_system_cpu_info
90 op_query_permission
91 op_revoke_permission
92 op_request_permission
93 op_open_plugin
94 op_run
95 op_run_status
96 op_kill
97 op_signal_bind
98 op_signal_unbind
99 op_signal_poll
100 op_start_tls
101 op_connect_tls
102 op_listen_tls
103 op_accept_tls
104 op_set_raw
105 op_isatty
106 op_console_size
107 op_ws_check_permission
108 op_ws_create
109 op_ws_send
110 op_ws_close
111 op_ws_next_event
112 op_apply_source_map
113 op_format_diagnostic
114 op_compile
115 op_transpile
--
That was all about ops registration. Let's move on to module evaluation.