Building a new Static Site Generator

I’ve been working on a project recently, which I’ve code-named docgen. It’s a static site generator, similar to jekyll, but with a few tweaks.

No Templating Language - (not exactly, but you’ll see what I mean below)
Javascript Based
Flexible

Motivation

I’ve used jekyll for the past five years (since I started this website) but there’s certain classes of things I’d like to do that jekyll doesn’t support very nicely.

sharing of scope between templates / includes.
tag pages. the ability to fork a template’s rendering into separate files.

To solve this, I’m working on this project. It’s not complete yet, but once I reach the point where I can port my site to use it, I’ll mark it as a the 1.0 release.

How does it work?

Docgen works by embedding spidermonkey and other components from the servo project to create an environment for rendering pages. Since we are embedding the whole engine, and not using eval() or manually parsing the template, we gain a bit more flexibility in the kinds of things we can do.

What is it?

You can read more about it on github, but here’s a quick overview of how the templating works.

All templates for docgen are valid html files. Technically liquid templates also are, but they have additional text nodes that contain the template logic. In this template language, the templating is built into the DOM tree.

The basis of all the templates is javascript. docgen will run any script tag with the static attribute set and remove it from the final dom tree.

<!-- This is a script, run at compile time. -->
<script static>
name = 'docgen'
profileUrl = 'https://www.richinfante.com'
items = ['a', 'b', 'c', 'd']
flag = true
</script>

You can substitute variables using standard mustache notation:

<!-- Variable Substitution -->
<div>{% raw %}{{name}}{% endraw %}</div>

Attribute substitution syntax (borrowed from vuejs). All content in a : prefixed attribute is executed in the javascript context and stringified.

<!-- Attribute Substitution -->
<a :href="profileUrl">My Profile</a>

Conditional rendering works more similar to how it does in vuejs as well. There is one nice feature gained here that I really like, which achieves the same idea as the optional-chaining proposal. However, instead of requiring special syntax, you can simply type obj.otherthing.propertyYouCareAbout even if obj or otherthing are undefined / null. We simply ignore exceptions for these issues, and return undefined.

Gone are the days of conditionals that look like this: obj && obj.otherthing && obj.otherthing.propertyYouCareAbout

<!-- Conditional Rendering -->
<div x-if="flag">Conditionally Rendered</div>

Iteration works a bit differently than you’ve probably seen before. The x-each attribute triggers loop rendering on a variable, however, you must also specify the name it is bound to using the x-as attribute. Additionally, the x-index attribute is available to allow you to bind a variable with the current loop index.

<!-- Iteration -->
<div x-each="items" x-as="letter">{% raw %}{{letter}}{% endraw %}</div>

Partials work as you might expect. You can specify a filename in the x-include attribute, and it will be inlined into the element’s contents.

Inside the partial, the parent global variable is set, containing the including file’s global scope.

<!-- Partials -->
<!-- Include another HTML file in the page -->
<div x-include="other_file.html"></div>

When a template is finished rendering, the engine checks for a global variable named layout. If it finds one, it will render that file as a template, and looks for a block like the one below. When (and if) it finds it, the child template’s <body> content is injected inside the marked <div>.

<!-- (Experimental) slots -->
<!-- When a child sets the `layout` variable, -->
<!-- The content is rendered inside the specified file -->
<!-- This is the marker that is replaced in the layout file -->
<div x-content-slot></div>

Other Dev. Notes

Segmentation Fault 11.

For a period of a day or so, I couldn’t get certain spidermonkey functions to work properly. I could read properties and evaluate scripts, but when trying to run JS_CallFunction or JS_SetProperty I’d recieve a fun “segmentation fault 11” message. rust-lldb was not to helpful for this, and the source readout it prints seemed to be unrelated to the issue I was experiencing.

For other people’s sanity, I’m writing the solution I found here in hope it helps you.

After setting up the global object, you need to add the following line:

//  rooted!(in(cx) let global = JS_NewGlobalObject(...));
let _ac = mozjs::jsapi::JSAutoCompartment::new(cx, global.get());

Code Snippets for servo + rust

Due to a lack of documentation because mozjs and much of the servo project is unstable, I had to resort to the C++ documentation and attempting to translate it. I’ve reproduced the most valuable snippets below for things I use a lot inside of docgen.

Set property

// here, `global` is the global object.
let prop_name = std::ffi::CString::new("child").unwrap();
let prop_name_ptr = prop_name.as_ptr() as *const i8;

// The value to set, converted to an JSObject. `value` has the type `JSVal`.
rooted!(in(cx) let val = value.to_object());
mozjs::rust::wrappers::JS_SetProperty(
    cx,
    global.handle(),
    prop_name_ptr,
    val.handle(),
);

Get Property

// here, the `iteration_value` is a JSVal representing the result of calling next() on an iterator.
let c_str = std::ffi::CString::new("value").unwrap();
let ptr = c_str.as_ptr() as *const i8;
rooted!(in(cx) let iteration_value_obj = iteration_value.to_object());
rooted!(in(cx) let mut iteration_result_value = UndefinedValue());
mozjs::rust::wrappers::JS_GetProperty(
    cx,
    iteration_value_obj.handle(),
    ptr,
    iteration_result_value.handle_mut(),
);

Function Calls

// This is a function call example (with no arguments).
// The `iter_result` object is the result of calling arr[Symbol.iterator]()
let next_str = std::ffi::CString::new("next").unwrap();
let next_ptr = next_str.as_ptr() as *const i8;
let args = mozjs::jsapi::HandleValueArray::new();
rooted!(in(cx) let mut iteration_value = UndefinedValue());
mozjs::rust::wrappers::JS_CallFunctionName(
    cx,
    iter_result.handle(),
    next_ptr,
    &args,
    iteration_value.handle_mut(),
);

Rich Infante