TM014 Module Loading

This is an incomplete draft

Module Compilation and Loading

This is a specification of the internal interfaces used for module loading. We should be able to use this to:

• Plan and implement different kinds of module loading (e.g. shared-gdrive, my-gdrive, files, builtins, future things like import-from-url), and keep semantics consistent
• Write user-facing docs to help folks understand in what order modules will load and run
• Figure out when and how to report compilation failures and type errors (with e.g. information about the import path that got us to the module that failed to compile)
• Have sensible semantics for when it is OK to cache compiled results and avoid recompilation

Locators

A main abstraction in the compilation is a Locator, which is a stateful interface to an abstract location that holds a Pyret module and can store a (serialized) compiled Pyret module. It can be used to get metadata about the module, like its name, dependencies, and exports.

The identitiy of a module is based on the URI that a locator specifies. In the case of multiple aliases for the same module (e.g. symlinks on disk), the URI of the locator decides whether the underlying modules should both be instatiated or treated identically.

  Locator :: {
    is-dirty :: -> Bool

    get-module :: -> PyretString
    get-dependencies :: -> Set<Dependency>
    get-provides :: -> Set<Provides>

    uri :: -> URI
    name :: -> String

    set-compiled :: CompileResult -> Undef

    // may throw an exception if no compiled version of the program is found
    get-compiled :: -> CompileResult

    // Note that CompileResults can contain both errors and successful
    // compilations
  }

Locators are needed both at compile time and at link time. At compile time, they will store compiled code using set-compiled, and at link time, the module-loading process will rely on get-compiled returning the last-compiled result for the module.

In order to map from programs and their metadata to actual code (whether compiled or not), we also need a function that knows what locator to use for each import statement and compilation job. The CompileContext can hold things like the root directory of the project, or the credentials of a user logged in to Google Drive, and can be used along with the Dependency (which is derived from the import statement), to get an unambiguous Locator for a module.

find-module :: CompileContext x Dependency -> Locator

Once modules are compiled, they need to have the compiled code registered with the running system. Compiled modules refer to their dependents by URI, and modules are loaded into the runtime by mapping their URI to their compiled code. Very concretely, in JavaScript, this will mean defining modules (probably in RequireJS) with strings mapping to JavaScript closures created by calling "eval" on compiled JavaScript source.

Registering a module does not cause its body to actually be evaluated (or that of its dependents). It simply makes it available for import.

To actually run a main module, a client of the runtime can call instantiate-module with a URI to run, and get back the result of evaluating that module's body (including test case results, exports for the REPL, etc).

PyretModuleVal = {
  values: ... values ...
  types: ... types ...
  answer: ... answer ...
  check-results: ... check-results ...
}

register-module :: Runtime x URI x CompileResult -> Undef
# Causes Runtime to link the CompileResult and store it at URI

instantiate-module :: Runtime x URI -> PyretModuleVal