4.3 Encode and decode

Overview

Encode and Decode represent a fundamental pair of functions within Deno's framework, serving as integral OPs (operations) and APIs (application programming interfaces). These functions hold substantial significance, facilitating the seamless transformation of V8 strings into V8 buffers and reciprocally. Notably, it's essential to grasp that a V8 string deviates slightly from a JS (JavaScript) string due to its construction as a C++ data structure. Although most external references pertain to infrastructural tasks, certain exceptions such as 'print' and 'get promise details' stand out.

Evident from their nomenclature, the 'encode' function assumes the role of translating a string into a sequence of bytes, effectively encapsulating the information within a binary format. Conversely, the 'decode' function specializes in the inverse process, adeptly converting bytes back into a human-readable string format. Both of these functions interact with V8 data structures, further emphasizing their underlying mechanics.

It's intriguing to highlight that the implementation of the encode and decode functions is rooted in Rust. Despite this Rust foundation, these functions seamlessly interact with V8 data structures, harmonizing the low-level Rust components with the intricacies of V8's C++ data structure. This harmonious interplay is pivotal in ensuring the efficient and accurate translation between V8 strings and buffers.

Both encode and decode are implemented in Deno core.

Functions

Registration

The process of registering the foundational OP functions takes place during startup, similar to how all other OPs are registered.

pub fn op_encode<'a>(
  scope: &mut v8::HandleScope<'a>,
  text: v8::Local<'a, v8::Value>,
) -> Result<v8::Local<'a, v8::Uint8Array>, Error>

pub fn op_decode<'a>(
  scope: &mut v8::HandleScope<'a>,
  #[buffer] zero_copy: &[u8],
) -> Result<v8::Local<'a, v8::String>, Error>

JS Space

The encode function is invoked from multiple locations within the Deno codebase. This utilization extends to the JavaScript context as well. Several illustrative instances encompass:

// fetch request body encoding

reqBody = typeof reqBody === "string" ? core.encode(reqBody) : reqBody;

// form data boundary encoding

this.boundaryChars = core.encode(this.boundary);

// Text encoder

encode(input = "") {
    webidl.assertBranded(this, TextEncoderPrototype);
    // The WebIDL type of `input` is `USVString`, but `core.encode` already
    // converts lone surrogates to the replacement character.
    input = webidl.converters.DOMString(
      input,
      "Failed to execute 'encode' on 'TextEncoder'",
      "Argument 1",
    );
    return core.encode(input);
}

Likewise, the core.decode functions are invoked from various points within the codebase. A few illustrative instances include:

// prompt 
return core.decode(new Uint8Array(buf));

// fetch response
return core.decode(buffer);

// reading file as text
return core.decode(buffer);

Rust space

Encode

Let's take a look at the code snippet from the OP that presents the 'encode' function.

pub fn op_encode<'a>(
  scope: &mut v8::HandleScope<'a>,
  text: v8::Local<'a, v8::Value>,
) -> Result<v8::Local<'a, v8::Uint8Array>, Error> {
  let text = v8::Local::<v8::String>::try_from(text)
    .map_err(|_| type_error("Invalid argument"))?;
  let text_str = serde_v8::to_utf8(text, scope);
  let bytes = text_str.into_bytes();
  let len = bytes.len();
  let backing_store =
    v8::ArrayBuffer::new_backing_store_from_vec(bytes).make_shared();
  let buffer = v8::ArrayBuffer::with_backing_store(scope, &backing_store);
  let u8array = v8::Uint8Array::new(scope, buffer, 0, len).unwrap();
  Ok(u8array)
}

The function is big, but it carries out a simple task:

• Convert string to bytes

• Set the return value (rv) so that v8 can proceed after the encode returns

Here is an example of an object and the encoded bytes:

// OBJECT TO ENCODE

{ path: "/var/tmp/test.log", len: 0, promiseId: 2 }

// ENCODED BYTES

[
  123,  34, 112,  97, 116, 104,  34,  58,  34,  47,
  118,  97, 114,  47, 116, 109, 112,  47, 116, 101,
  115, 116,  46, 108, 111, 103,  34,  44,  34, 108,
  101, 110,  34,  58,  48,  44,  34, 112, 114, 111,
  109, 105, 115, 101,  73, 100,  34,  58,  50, 125
]

Decode

The code for the decode function is also equally simple:

pub fn op_decode<'a>(
  scope: &mut v8::HandleScope<'a>,
  #[buffer] zero_copy: &[u8],
) -> Result<v8::Local<'a, v8::String>, Error> {
  let buf = &zero_copy;

  match v8::String::new_from_utf8(scope, buf, v8::NewStringType::Normal) {
    Some(text) => Ok(text),
    None => Err(range_error("string too long")),
  }
}

• Get buffer

• Convert buffer to string

• Set return value so that v8 can proceed after the decode returns

Here is an example of bytes and the decode object:

// BYTES

[
  123,  34, 111, 107,  34,  58,
  123, 125,  44,  34, 112, 114,
  111, 109, 105, 115, 101,  73,
  100,  34,  58,  50, 125
]


// DECODED OBJECT

{ ok: {}, promiseId: 2 }