Operations, often referred to as "Ops," play a crucial role in Deno. Ops act as the primary connectors between JavaScript and Rust, effectively forming a bridge between the two. This interaction happens back and forth, allowing seamless communication. Ops are essential because V8 strictly adheres to the ECMAScript specification, the standard for JavaScript. Any functionality that goes beyond this specification must be provided as an external reference. Even fundamental actions like using console.log require an external op call, as the ECMAScript specification doesn't encompass console output.
In the previous chapter, we explored how external references are registered with V8. We will now quickly revisit that concept. Additionally, we will look into built-in and ext ops. This section will specifically focus on the registration process for ops. Later on, we will examine how ops are triggered during runtime and how the system manages their responses.
Overview
In short:
Ops are low-level functions that are implemented in Rust to support the high-level functions in JS.
For example, crypto.randomUUID is a high-level function, and the corresponding Rust op is implemented by op_crypto_random_uuid.
pub fn op_crypto_random_uuid(state: &mut OpState) -> Result<String, AnyError> {
let maybe_seeded_rng = state.try_borrow_mut::<StdRng>();
let uuid = if let Some(seeded_rng) = maybe_seeded_rng {
let mut bytes = [0u8; 16];
seeded_rng.fill(&mut bytes);
uuid::Builder::from_bytes(bytes)
.with_version(uuid::Version::Random)
.into_uuid()
} else {
uuid::Uuid::new_v4()
};
Ok(uuid.to_string())
}
In Deno, operations (OPs) typically have a one-to-one mapping. This means that one JavaScript API directly corresponds to one Rust operation (OP). All the operations (OPs) follow a consistent pattern, from the initial setup to processing the result. The following outlines the typical phases of an operation (OP):
Deno operations progress through various stages. The diagram above illustrates the standard operation state machine. Let's examine each state:
Register handler
In this step, an operation handler is registered.
Waiting to get called
This state resembles an idle condition where the operation awaits a call.
Dispatch
Dispatch involves moving across the bridge from V8 to Deno.
Execute
Deno executes the operation's handler.
Process result
The outcome is sent back to V8, where it undergoes processing in JavaScript.
This step is akin to returning from the bridge.
The process of dispatching and result processing happens during runtime. We will discuss the details later. For now, our focus is on registering operations.
Types of Ops
All the operations can be divided into two main categories:
Sync Operations:
In sync operations, the request and response are processed one after the other, in a step-by-step manner.
During the execution of sync operations, the process is blocked, meaning it waits for each step to complete before moving forward.
Async Operations:
Async operations, on the other hand, handle requests and responses asynchronously, which means they can process multiple tasks concurrently.
The execution of async operations is not blocked, allowing the program to continue with other tasks while waiting for certain operations to finish.
Certain operations fall distinctly into either of these categories, while some operations can belong to both. For instance, operations like getting and setting environment variables are categorized as sync operations. Another example involves reading a file, which can involve both sync and async operations, depending on the specific operation and the user's choice.
The suitability of async operations varies based on the nature of the operation. Some operations are better suited for sync processing, and thus only have sync variants. On the other hand, for operations like reading a file, the decision to use sync or async mode is up to the user. If dealing with a particularly large file, the user might opt for async reading to improve efficiency.
Registration
Op registration happens at startup. Deno registers all of its low-level ops right at the process startup. There two sources of OPs:
Built-in OPs: These OPs are part of Deno core. These cover the most basic & fundamental OPs. Some examples are - op_print, op_resources, op_read, op_write, op_encode, op_decode, etc.
EXT OPs: These OPs come from Deno's ext modules. As we know that the ext modules are a close combination of JS APIs alongwith their low-level OPs. The OPs coming from EXT usually have the 1:1 mapping from JS API to the OP. Some examples are - op_fetch, op_crypto_sign_key, op_crypto_get_random_values, etc.
The code in the main worker below registers all the OPs with V8 as external references:
pub fn from_options(
main_module: ModuleSpecifier,
permissions: PermissionsContainer,
mut options: WorkerOptions,
) -> Self {
deno_core::extension!(deno_permissions_worker,
options = {
permissions: PermissionsContainer,
unstable: bool,
enable_testing_features: bool,
},
state = |state, options| {
state.put::<PermissionsContainer>(options.permissions);
state.put(ops::UnstableChecker { unstable: options.unstable });
state.put(ops::TestingFeaturesEnabled(options.enable_testing_features));
},
);
// Permissions: many ops depend on this
let unstable = options.bootstrap.unstable;
let enable_testing_features = options.bootstrap.enable_testing_features;
let exit_code = ExitCode(Arc::new(AtomicI32::new(0)));
let create_cache = options.cache_storage_dir.map(|storage_dir| {
let create_cache_fn = move || SqliteBackedCache::new(storage_dir.clone());
CreateCache(Arc::new(create_cache_fn))
});
// NOTE(bartlomieju): ordering is important here, keep it in sync with
// `runtime/build.rs`, `runtime/web_worker.rs` and `cli/build.rs`!
let mut extensions = vec![
// Web APIs
deno_webidl::deno_webidl::init_ops_and_esm(),
deno_console::deno_console::init_ops_and_esm(),
deno_url::deno_url::init_ops_and_esm(),
deno_web::deno_web::init_ops_and_esm::<PermissionsContainer>(
options.blob_store.clone(),
options.bootstrap.location.clone(),
),
deno_fetch::deno_fetch::init_ops_and_esm::<PermissionsContainer>(
deno_fetch::Options {
user_agent: options.bootstrap.user_agent.clone(),
root_cert_store_provider: options.root_cert_store_provider.clone(),
unsafely_ignore_certificate_errors: options
.unsafely_ignore_certificate_errors
.clone(),
file_fetch_handler: Rc::new(deno_fetch::FsFetchHandler),
..Default::default()
},
),
deno_cache::deno_cache::init_ops_and_esm::<SqliteBackedCache>(
create_cache,
),
deno_websocket::deno_websocket::init_ops_and_esm::<PermissionsContainer>(
options.bootstrap.user_agent.clone(),
options.root_cert_store_provider.clone(),
options.unsafely_ignore_certificate_errors.clone(),
),
deno_webstorage::deno_webstorage::init_ops_and_esm(
options.origin_storage_dir.clone(),
),
deno_crypto::deno_crypto::init_ops_and_esm(options.seed),
deno_broadcast_channel::deno_broadcast_channel::init_ops_and_esm(
options.broadcast_channel.clone(),
unstable,
),
deno_ffi::deno_ffi::init_ops_and_esm::<PermissionsContainer>(unstable),
deno_net::deno_net::init_ops_and_esm::<PermissionsContainer>(
options.root_cert_store_provider.clone(),
unstable,
options.unsafely_ignore_certificate_errors.clone(),
),
deno_tls::deno_tls::init_ops_and_esm(),
deno_kv::deno_kv::init_ops_and_esm(
MultiBackendDbHandler::remote_or_sqlite::<PermissionsContainer>(
options.origin_storage_dir.clone(),
),
unstable,
),
deno_napi::deno_napi::init_ops_and_esm::<PermissionsContainer>(),
deno_http::deno_http::init_ops_and_esm::<DefaultHttpPropertyExtractor>(),
deno_io::deno_io::init_ops_and_esm(Some(options.stdio)),
deno_fs::deno_fs::init_ops_and_esm::<PermissionsContainer>(
unstable,
options.fs.clone(),
),
deno_node::deno_node::init_ops_and_esm::<PermissionsContainer>(
options.npm_resolver,
options.fs,
),
// Ops from this crate
ops::runtime::deno_runtime::init_ops_and_esm(main_module.clone()),
ops::worker_host::deno_worker_host::init_ops_and_esm(
options.create_web_worker_cb.clone(),
options.format_js_error_fn.clone(),
),
ops::fs_events::deno_fs_events::init_ops_and_esm(),
ops::os::deno_os::init_ops_and_esm(exit_code.clone()),
ops::permissions::deno_permissions::init_ops_and_esm(),
ops::process::deno_process::init_ops_and_esm(),
ops::signal::deno_signal::init_ops_and_esm(),
ops::tty::deno_tty::init_ops_and_esm(),
ops::http::deno_http_runtime::init_ops_and_esm(),
deno_permissions_worker::init_ops_and_esm(
permissions,
unstable,
enable_testing_features,
),
runtime::init_ops_and_esm(),
];
for extension in &mut extensions {
#[cfg(not(feature = "__runtime_js_sources"))]
{
extension.js_files = std::borrow::Cow::Borrowed(&[]);
extension.esm_files = std::borrow::Cow::Borrowed(&[]);
extension.esm_entry_point = None;
}
#[cfg(feature = "__runtime_js_sources")]
{
for source in extension.esm_files.to_mut() {
maybe_transpile_source(source).unwrap();
}
for source in extension.js_files.to_mut() {
maybe_transpile_source(source).unwrap();
}
}
}
extensions.extend(std::mem::take(&mut options.extensions));
#[cfg(all(feature = "include_js_files_for_snapshotting", feature = "dont_create_runtime_snapshot", not(feature = "__runtime_js_sources")))]
options.startup_snapshot.as_ref().expect("Sources are not embedded, snapshotting was disabled and a user snapshot was not provided.");
// Clear extension modules from the module map, except preserve `node:*`
// modules.
let preserve_snapshotted_modules =
Some(SUPPORTED_BUILTIN_NODE_MODULES_WITH_PREFIX);
let mut js_runtime = JsRuntime::new(RuntimeOptions {
module_loader: Some(options.module_loader.clone()),
startup_snapshot: options
.startup_snapshot
.or_else(crate::js::deno_isolate_init),
create_params: options.create_params,
source_map_getter: options.source_map_getter,
get_error_class_fn: options.get_error_class_fn,
shared_array_buffer_store: options.shared_array_buffer_store.clone(),
compiled_wasm_module_store: options.compiled_wasm_module_store.clone(),
extensions,
preserve_snapshotted_modules,
inspector: options.maybe_inspector_server.is_some(),
is_main: true,
..Default::default()
});
if let Some(server) = options.maybe_inspector_server.clone() {
server.register_inspector(
main_module.to_string(),
&mut js_runtime,
options.should_break_on_first_statement
|| options.should_wait_for_inspector_session,
);
// Put inspector handle into the op state so we can put a breakpoint when
// executing a CJS entrypoint.
let op_state = js_runtime.op_state();
let inspector = js_runtime.inspector();
op_state.borrow_mut().put(inspector);
}
let bootstrap_fn_global = {
let context = js_runtime.main_context();
let scope = &mut js_runtime.handle_scope();
let context_local = v8::Local::new(scope, context);
let global_obj = context_local.global(scope);
let bootstrap_str =
v8::String::new_external_onebyte_static(scope, b"bootstrap").unwrap();
let bootstrap_ns: v8::Local<v8::Object> = global_obj
.get(scope, bootstrap_str.into())
.unwrap()
.try_into()
.unwrap();
let main_runtime_str =
v8::String::new_external_onebyte_static(scope, b"mainRuntime").unwrap();
let bootstrap_fn =
bootstrap_ns.get(scope, main_runtime_str.into()).unwrap();
let bootstrap_fn =
v8::Local::<v8::Function>::try_from(bootstrap_fn).unwrap();
v8::Global::new(scope, bootstrap_fn)
};
Self {
js_runtime,
should_break_on_first_statement: options.should_break_on_first_statement,
should_wait_for_inspector_session: options
.should_wait_for_inspector_session,
exit_code,
bootstrap_fn_global: Some(bootstrap_fn_global),
}
}
Let's look at the master function that register all the Ops with V8:
pub(crate) fn external_references(
ops: &[OpCtx],
additional_references: &[v8::ExternalReference],
) -> v8::ExternalReferences {
// Overallocate a bit, it's better than having to resize the vector.
let mut references =
Vec::with_capacity(4 + (ops.len() * 4) + additional_references.len());
references.push(v8::ExternalReference {
function: call_console.map_fn_to(),
});
references.push(v8::ExternalReference {
function: import_meta_resolve.map_fn_to(),
});
references.push(v8::ExternalReference {
function: catch_dynamic_import_promise_error.map_fn_to(),
});
references.push(v8::ExternalReference {
function: empty_fn.map_fn_to(),
});
for ctx in ops {
let ctx_ptr = ctx as *const OpCtx as _;
references.push(v8::ExternalReference { pointer: ctx_ptr });
references.push(v8::ExternalReference {
function: ctx.decl.v8_fn_ptr,
});
if let Some(fast_fn) = &ctx.decl.fast_fn {
references.push(v8::ExternalReference {
pointer: fast_fn.function as _,
});
references.push(v8::ExternalReference {
pointer: ctx.fast_fn_c_info.unwrap().as_ptr() as _,
});
}
}
references.extend_from_slice(additional_references);
let refs = v8::ExternalReferences::new(&references);
// Leak, V8 takes ownership of the references.
std::mem::forget(references);
refs
}
Although the code of OPs is in Rust, they're invoked from JS space. Therefore, each OP has a corresponding registration function in JS too. This also comes from Deno core.
Deno.__op__registerOp = function (isAsync, op, opName) {
const core = Deno.core;
if (isAsync) {
if (core.ops[opName] !== undefined) {
return;
}
core.asyncOps[opName] = op;
const fn = function (...args) {
if (this !== core.ops) {
// deno-lint-ignore prefer-primordials
throw new Error(
"An async stub cannot be separated from Deno.core.ops. Use ???",
);
}
return core.asyncStub(opName, args);
};
fn.name = opName;
core.ops[opName] = fn;
} else {
core.ops[opName] = op;
}
};
The call to registerOp called from Rust code:
pub(crate) fn initialize_context<'s>(
scope: &mut v8::HandleScope<'s>,
context: v8::Local<'s, v8::Context>,
op_ctxs: &[OpCtx],
init_mode: InitMode,
) -> v8::Local<'s, v8::Context> {
let global = context.global(scope);
let mut codegen = String::with_capacity(op_ctxs.len() * 200);
codegen.push_str(include_str!("bindings.js"));
_ = writeln!(
codegen,
"Deno.__op__ = function(opFns, callConsole, console) {{"
);
if init_mode == InitMode::New {
_ = writeln!(codegen, "Deno.__op__console(callConsole, console);");
}
for op_ctx in op_ctxs {
if op_ctx.decl.enabled {
_ = writeln!(
codegen,
"Deno.__op__registerOp({}, opFns[{}], \"{}\");",
op_ctx.decl.is_async, op_ctx.id, op_ctx.decl.name
);
} else {
_ = writeln!(
codegen,
"Deno.__op__unregisterOp({}, \"{}\");",
op_ctx.decl.is_async, op_ctx.decl.name
);
}
}
codegen.push_str("Deno.__op__cleanup();");
_ = writeln!(codegen, "}}");
let script = v8::String::new_from_one_byte(
scope,
codegen.as_bytes(),
v8::NewStringType::Normal,
)
.unwrap();
let script = v8::Script::compile(scope, script, None).unwrap();
script.run(scope);
let deno = get(scope, global, b"Deno", "Deno");
let op_fn: v8::Local<v8::Function> =
get(scope, deno, b"__op__", "Deno.__op__");
let recv = v8::undefined(scope);
let op_fns = v8::Array::new(scope, op_ctxs.len() as i32);
for op_ctx in op_ctxs {
let op_fn = op_ctx_function(scope, op_ctx);
op_fns.set_index(scope, op_ctx.id as u32, op_fn.into());
}
if init_mode == InitMode::FromSnapshot {
op_fn.call(scope, recv.into(), &[op_fns.into()]);
} else {
// Bind functions to Deno.core.*
let call_console_fn = v8::Function::new(scope, call_console).unwrap();
// Bind v8 console object to Deno.core.console
let extra_binding_obj = context.get_extras_binding_object(scope);
let console_obj: v8::Local<v8::Object> = get(
scope,
extra_binding_obj,
b"console",
"ExtrasBindingObject.console",
);
op_fn.call(
scope,
recv.into(),
&[op_fns.into(), call_console_fn.into(), console_obj.into()],
);
}
context
}
At the time of writing, there are 478 ops that get registered. Here is the complete list:
