6.5 Registration and instantiation

Overview

All modules are prepared to enter v8. This occurs in two steps:

1. Compiling and registering the module.

2. Instantiating the module.

Finally, they are loaded into v8, ready for execution or evaluation.

Registration

Let's review how registration works because we have some imports in this example. This time, the process will be a bit longer since the imports also need to be loaded into V8. Here's a detailed explanation:

1. load_main_module The first step involves loading the main module.

2. Recursive Loading and Preparation After that, the process performs recursive loading and prepares the modules for registration.

3. Loop Over Pending List and Module Registration Next, there is a loop that goes through a list of pending modules and registers them.

4. register_and_recurse Within the registration process, there's a loop that handles both registration and recursion.

5. Loop Over Imports Another loop iterates over all the imports of a module.

6. Adding Imports to Pending List During this iteration, if an import is not yet registered, it is added to the pending list for later processing.

The "load_module" loop continues registering modules until the pending list becomes empty. Here is an abbreviated code of all the three functions (only import specific code is shown, rest is suppressed):

pub async fn load_main_module(
    &self,
    isolate: &mut v8::Isolate,
    specifier: &ModuleSpecifier,
    code: Option<ModuleCode>,
  ) -> Result<ModuleId, Error> {
    let module_map_rc = self.0.module_map();
    if let Some(code) = code {
      let specifier = specifier.as_str().to_owned().into();
      let scope = &mut self.handle_scope(isolate);
      // true for main module
      module_map_rc
        .borrow_mut()
        .new_es_module(scope, true, specifier, code, false)
        .map_err(|e| match e {
          ModuleError::Exception(exception) => {
            let exception = v8::Local::new(scope, exception);
            exception_to_err_result::<()>(scope, exception, false).unwrap_err()
          }
          ModuleError::Other(error) => error,
        })?;
    }

    let mut load =
      ModuleMap::load_main(module_map_rc.clone(), &specifier).await?;

    while let Some(load_result) = load.next().await {
      let (request, info) = load_result?;
      let scope = &mut self.handle_scope(isolate);
      load.register_and_recurse(scope, &request, info).map_err(
        |e| match e {
          ModuleError::Exception(exception) => {
            let exception = v8::Local::new(scope, exception);
            exception_to_err_result::<()>(scope, exception, false).unwrap_err()
          }
          ModuleError::Other(error) => error,
        },
      )?;
    }

    let root_id = load.root_module_id.expect("Root module should be loaded");
    self.instantiate_module(isolate, root_id).map_err(|e| {
      let scope = &mut self.handle_scope(isolate);
      let exception = v8::Local::new(scope, e);
      exception_to_err_result::<()>(scope, exception, false).unwrap_err()
    })?;
    Ok(root_id)
}

pub(crate) fn register_and_recurse(
    &mut self,
    scope: &mut v8::HandleScope,
    module_request: &ModuleRequest,
    module_source: ModuleSource,
  ) -> Result<(), ModuleError> {
    let expected_asserted_module_type = module_source.module_type.into();
    let module_url_found = module_source.module_url_found;
    let module_url_specified = module_source.module_url_specified;

    if module_request.asserted_module_type != expected_asserted_module_type {
      return Err(ModuleError::Other(generic_error(format!(
        "Expected a \"{}\" module but loaded a \"{}\" module.",
        module_request.asserted_module_type, module_source.module_type,
      ))));
    }

    // Register the module in the module map unless it's already there. If the
    // specified URL and the "true" URL are different, register the alias.
    let module_url_found = if let Some(module_url_found) = module_url_found {
      let (module_url_found1, module_url_found2) =
        module_url_found.into_cheap_copy();
      self.module_map_rc.borrow_mut().alias(
        module_url_specified,
        expected_asserted_module_type,
        module_url_found1,
      );
      module_url_found2
    } else {
      module_url_specified
    };

    let maybe_module_id = self
      .module_map_rc
      .borrow()
      .get_id(&module_url_found, expected_asserted_module_type);
    let module_id = match maybe_module_id {
      Some(id) => {
        debug!(
          "Already-registered module fetched again: {:?}",
          module_url_found
        );
        id
      }
      None => match module_source.module_type {
        ModuleType::JavaScript => {
          self.module_map_rc.borrow_mut().new_es_module(
            scope,
            self.is_currently_loading_main_module(),
            module_url_found,
            module_source.code,
            self.is_dynamic_import(),
          )?
        }
        ModuleType::Json => self.module_map_rc.borrow_mut().new_json_module(
          scope,
          module_url_found,
          module_source.code,
        )?,
      },
    };

    // Recurse the module's imports. There are two cases for each import:
    // 1. If the module is not in the module map, start a new load for it in
    //    `self.pending`. The result of that load should eventually be passed to
    //    this function for recursion.
    // 2. If the module is already in the module map, queue it up to be
    //    recursed synchronously here.
    // This robustly ensures that the whole graph is in the module map before
    // `LoadState::Done` is set.
    let mut already_registered = VecDeque::new();
    already_registered.push_back((module_id, module_request.clone()));
    self.visited.insert(module_request.clone());
    while let Some((module_id, module_request)) = already_registered.pop_front()
    {
      let referrer = ModuleSpecifier::parse(&module_request.specifier).unwrap();
      let imports = self
        .module_map_rc
        .borrow()
        .get_requested_modules(module_id)
        .unwrap()
        .clone();
      for module_request in imports {
        if !self.visited.contains(&module_request)
          && !self
            .visited_as_alias
            .borrow()
            .contains(&module_request.specifier)
        {
          if let Some(module_id) = self.module_map_rc.borrow().get_id(
            module_request.specifier.as_str(),
            module_request.asserted_module_type,
          ) {
            already_registered.push_back((module_id, module_request.clone()));
          } else {
            let request = module_request.clone();
            let specifier =
              ModuleSpecifier::parse(&module_request.specifier).unwrap();
            let visited_as_alias = self.visited_as_alias.clone();
            let referrer = referrer.clone();
            let loader = self.loader.clone();
            let is_dynamic_import = self.is_dynamic_import();
            let fut = async move {
              // `visited_as_alias` unlike `visited` is checked as late as
              // possible because it can only be populated after completed
              // loads, meaning a duplicate load future may have already been
              // dispatched before we know it's a duplicate.
              if visited_as_alias.borrow().contains(specifier.as_str()) {
                return Ok(None);
              }
              let load_result = loader
                .load(&specifier, Some(&referrer), is_dynamic_import)
                .await;
              if let Ok(source) = &load_result {
                if let Some(found_specifier) = &source.module_url_found {
                  visited_as_alias
                    .borrow_mut()
                    .insert(found_specifier.as_str().to_string());
                }
              }
              load_result.map(|s| Some((request, s)))
            };
            self.pending.push(fut.boxed_local());
          }
          self.visited.insert(module_request);
        }
      }
    }

    // Update `self.state` however applicable.
    if self.state == LoadState::LoadingRoot {
      self.root_module_id = Some(module_id);
      self.root_asserted_module_type = Some(module_source.module_type.into());
      self.state = LoadState::LoadingImports;
    }
    if self.pending.is_empty() {
      self.state = LoadState::Done;
    }

    Ok(())
  }
}

The process is pretty simple. It's similar to creating the module graph. But, to ensure everything is covered, we'll review all the steps that lead to registering modules for the hello world v2 program.

Step 1

Compile and register module file:///Users/mayankc/Work/source/denoExamples/helloV2.ts.

ID	Module
1	file:///Users/mayankc/Work/source/denoExamples/helloLogV2.ts

Step 2

Go through imports and add them to the pending list

• add https://deno.land/x/machine_id/mod.ts to the pending list

• add npm:nanoid to the pending list

Step 3

Compile and register module https://deno.land/x/machine_id/mod.ts.

ID	Module
2	https://deno.land/x/machine_id/mod.ts

There are further external dependencies from machine_id module.

Step 4

Compile and register module https://deno.land/x/machine_id/mod.ts.

ID	Module
3	npm:nanoid or file:///Users/mayankc/Work/source/denoExamples/node_modules/.deno/nanoid@4.0.2/node_modules/nanoid/index.js

Step 5

Go through imports and add them to the pending list

• add file:///Users/mayankc/Work/source/denoExamples/node_modules/.deno/nanoid@4.0.2/node_modules/nanoid/url-alphabet/index.js to the pending list

Step 6

Compile and register module file:///Users/mayankc/Work/source/denoExamples/node_modules/.deno/nanoid@4.0.2/node_modules/nanoid/url-alphabet/index.js.

ID	Module
4	file:///Users/mayankc/Work/source/denoExamples/node_modules/.deno/nanoid@4.0.2/node_modules/nanoid/url-alphabet/index.js

Step 7

There are no further imports, nothing gets added to the pending list.

Step 8

Since the pending list is empty, the registration process is complete. All the modules that were imported from the main module have been compiled and registered.

It's crucial to understand that these modules have been loaded separately into V8, without any connection between them for now. The actual connection will occur when they are instantiated.

Instantiation

Regardless of how many imports or modules there are, instantiation only occurs in the root or main module. To remind you, here is the relevant code where "instantiate" is called.

let root_id = load.root_module_id.expect("Root module should be loaded");
self.instantiate_module(isolate, root_id).map_err(|e| {
  let scope = &mut self.handle_scope(isolate);
  let exception = v8::Local::new(scope, e);
  exception_to_err_result::<()>(scope, exception, false).unwrap_err()
})?;
Ok(root_id)

In the previous chapter, we learned about the mod_instantiate function. This function helps set up a module in v8, which is like a place where JavaScript code runs. In the last chapter's hello world program, we didn't need any callbacks because that program didn't use any external code.

But things are a bit different now. In this chapter, we're working with modules that have imports. An import is like a way to bring in code from another module. When we have imports, the v8 engine wants us to provide a callback to help it figure out where to find the imported modules.

Here's how it works:

1. We start by creating the main module.

2. For each import in each module:

   • V8 asks us for a module handle using a callback.

   • In this callback, V8 tells us which module it wants to find and the module that's asking for it. This helps V8 connect the dots correctly.

   • This process continues, going deeper into each module, until all the modules are taken care of.

It's like solving a puzzle one piece at a time. Each module is a piece, and V8 needs us to tell it where to find the pieces it's missing. This whole process happens in a loop, like a chain reaction, until everything is set up.

Remember, it's a looping process, so we don't have to set up each module individually. We just start with the main module, and V8 takes care of the rest, step by step.

Now, let's see how all of this works with our updated hello world v2 program.

Step 1

Instantiate main module id=1.

Step 2

Receive callback to resolve:

Resolve	Referrer
npm:nanoid	file:///Users/mayankc/Work/source/denoExamples/helloV2.ts

Step 3

Receive callback to resolve:

Resolve	Referrer
https://deno.land/x/machine_id/mod.ts	file:///Users/mayankc/Work/source/denoExamples/helloV2.ts

Step 4

Receive callback to resolve:

Resolve	Referrer
crypto	https://deno.land/x/machine_id/mod.ts

Yes, even built-in modules like crypto needs a resolve callback. They're built-in for us, but not for V8.

Step 5

Receive callback to resolve:

Resolve	Referrer
crypto	https://deno.land/x/machine_id/mod.ts

Step 6

Receive callback to resolve:

Resolve	Referrer
./url-alphabet/index.js	npm:nanoid or file:///Users/mayankc/Work/source/denoExamples/node_modules/.deno/nanoid@4.0.2/node_modules/nanoid/index.js

--

After all the import callbacks are successfully resolved, v8 completes the module setup. Both the main module and its imports are prepared and set up inside v8 through a recursive process. They are now set to be evaluated.

Before we move on to evaluation, let's understand how operations (ops) are registered. This is important because our hello world v2 program includes both synchronous and asynchronous ops. We've touched upon the ops registration briefly in a previous chapter. Before we proceed to execute the program, let's take a closer look at this registration process.