Plugins Programmer Manual

Stag heavily uses plugins. By itself it only visits files and writes the files to their output destination by either copying them or running through Jinja templating engine.^[1]. Everything else is managed by the plugins.

How plugins work

Plugins are small pieces of Python code which Stag runs when certain events occur. Usually they are used to generate or modify Stag’s output, but they can do virtually anything.

Do you want to extend Stag’s capabilities to read a custom file format? No problem. Do you want your plugin to run a GUI program which interactively asks users a questions about the Universe? That’s also fine.

When Stag visits a file, it doesn’t even open it. It only reads its path and fills a Path object, which is used to create a new Page object inside a Site. Created pages are available e.g. by inspecting site.pages, but you could also subscribe to the site.signals.page_added signal. Path created by Stag is available as page.source.

On the other hand, Jinja templating engine creates files from the page.output, which is initially uninitialised. It’s plugins job to go from a mere page.source to the page which has its output, metadata and all the other blocks created.

Important

One thing which must be remembered is this: there are no "virtual" files inside your content directory. All the files which Stag finds will be used to create a site in one way or the other. If nothing else works, Stag will simply copy them.

Creating a new plugin

Registering plugins

Creating a plugin is very simple. Just create a plugins directory inside your sites’s directory and then create a Python file with a function register_plugin(). This function is the entry point for your plugin. It will be called once by Stag after the plugin is imported and it receives an initialised Site object, which is a central object in Stag. Site holds a list of pages, handles of signals, site configuration and a bunch of other goodies.

Typically, register_plugin() is used to initialize the plugin, which usually ends up in connecting some of plugin functions to one or more available signals.

Warning

It’s best to avoid placing any executable code in plugin’s global scope. It’s possible to fool Python’s import system and to import the plugin twice, not once. On the other hand, it’s guaranteed that Stag will call register_plugin() only once.

Plugin example

Consider the following, not very useful plugin. File structure should look like this:

Site file structure

content/
  index.md
plugins/
  myplugin.py

config.toml

myplugin.py

def greet_user(site):
    print("Hello from the plugin!")


def register_plugin(site):
    site.signals.plugins_loaded.connect(greet_user)

That’s it! From now on, every time a site is built, user will be greeted by our plugin.

Note

Packages and modules

Above example keeps a plugin as a Python module. Stag will iterate these and try to import them.

Alternatively, you might choose to keep plugins as packages, which results in a single directory per plugin:

plugins/
  myplugin/
    __init__.py
    myplugin.py

You might prefer this scheme because it is easier to e.g. clone plugins from the internet this way.

URLs

Each Page has 2 immutable fields: base and path. Together they form an URL. base is common for all pages and is configured in config.toml url field. path is indirectly set by plugins which register a function which decides it.

path is unique and distinguishes page among the others. If two plugins would create two files from a single source, both files must be distinct pages, with own paths.

path also tells Stag the expected location of files:

when path doesn’t have an extension (like /post/my-post), it is considered a "pretty" url. Jinja renderer will create post/my-post/index.html file in that case
when path has an extension (like /post/my-post.html), Jinja renderer will create use it without changes.

If no plugin provides a specialised way to determine page’s path, Stag will reuse path from the source file.

To register a path provider function, plugins may use site.readers.register_reader. Path provider should accept a Path object and return deduced path (a string). Optionally, register_reader accepts a condition function which decides whether path provider is called at all.

Typically, returned paths are relative to the base and they start with a forward slash: /path, /other/path.html.

Path provider which creates pretty paths for all files

def deduce_url(path):
    if path.filebase == "index":
        return path.reldirname
    return os.path.join(path.reldirname, path.filebase)


def register_plugin(site):
    site.readers.register_reader(deduce_url, lambda p: p.ext == "md")

Plugin Configuration

Plugins can have a custom configuration tables in config.toml. Stag will read all of the entries inside the plugins section of config.toml anyway, and you can access them in convenient way, but it’s nice to do some validation of these. To do so, plugins can register a custom objects, whose attributes should reflect the names of keys used in config.toml. These objects might also supply other methods to these tables.

Note	it is advisable to use attrs module to create such objects. It is a dependency of Stag anyway.

import attr

@attr.s(auto_attribs=True)
class MyPluginCfg:
    myoption: str = "foo"
    other_option: List[str] = attr.Factory(lambda: ["foo", "bar", "baz"])


def register_plugin(site):
    site.config.update_plugin_table("myplugin", MyPluginCfg())

Signals

Because register_plugin() is called only once and probably there are a bunch of things which Stag hasn’t initialised, it’s not very useful to do anything besides plugin’s initialisation at that point. That’s why we have signals!

Built on top of observer pattern (duh!), they are called when certain events occur (see: List of signals). Basically, they are mere function calls. Once a signal is emitted (signal.emit(…)) it goes thhrough all of the subscribed observers and calls each one with the arguments of emit() call.

Connecting to signals

To connect a function to the signal, you must have a signal object and call connect(fn) on it.

To simplify things, Stag stores most global signals inside site.signals.

Warning

Weak references

By default observers are stored as weak references. It means that if they ever are deleted (e.g. temporaries which go out of scope), Stag won’t call them! It’s the best to not connect temporaries, but if you really want to do it, use weak=True parameter when connecting the observer.

def callback(*a):
    print("callback")

# OK
some_signal.connect(callback)

# OK
some_signal.connect((lambda *a: print("lambda 1", *a)), weak=True)

# NOT OK, lambda 2 won't be ever called
some_signal.connect(lambda *a: print("lambda 2", *a))

In addition to site.signals, some signals are sent by Pages themselves. You can connect to them like this:

def input_created(page, inp):
    assert page.input is inp
    print(f"input created for page {page.url}")


def page_cb(page):
    page.input_created.connect(input_created)


def register_plugin(site):
    site.signals.page_added.connect(page_cb)

Or more sparse:

def input_created(page, inp):
    assert page.input is inp
    print(f"input created for page {page.url}")


def register_plugin(site)
    site.signals.page_added.connect(
        (lambda p: p.input_created.connect(input_created)),
        weak=False)

List of signals

Table 1. signals.signals
Signal	Emit Parameters	Description
`signals.plugins_loaded`		Emitted immediately after all plugins are fully loaded and their `register_plugin()` functions are called.
`signals.site_finished`	`Site`	Called once site is fully generated and Stag is about to quit.
`signals.readers_init`	`Site`	Called before files visitation.
`signals.readers_finished`	`Site`	Called after files visitation. Usually at this point it is expected that all "reader" plugins are done, i.e. that they have created `input` and `metadata` for supported filetypes.
`signals.processors_init`	`Site`	A trigger for "processors" (plugins which generate pages' `output`)
`signals.processors_finished`	`Site`	Emitted immediately after `processors_init`, i.e. when all "processors" finished their jobs.
`signals.rendering_init`	`Site`	Emitted before rendering the site (i.e. copying static files and rendering templates).
`signals.rendering_finished`	`Site`	Emitted once after rendering finishes.
`signals.jinja_environment_prepared`	`jinja2.Environment, Site`	Emitted once Jinja Environment has been created. Environment passed as the argument will be used for rendering templates. User plugins might modify it, e.g. by adding custom filters or functions. For user convenience, `Site` is passed as well.

Table 2. Site signals
Signal	Emit Parameters	Description
`site.page_added`	`Page`	Emitted after `site` stores a new page (e.g. after `site.make_page()` or `site.get_or_make_page()` calls).

Table 3. Page signals
Signal	Emit Parameters	Description
`page.metadata_created`	`Page, Metadata`	Emitted when a new Metadata is created for this page.
`page.metadata_removed`	`Page, Metadata`	Emitted when Metadata is removed for this page.
`page.source_created`	`Page, Source`	Emitted when a new Source is created for this page.
`page.source_removed`	`Page, Source`	Emitted when Source is removed for this page.
`page.input_created`	`Page, Input`	Emitted when a new Input is created for this page.
`page.input_removed`	`Page, Input`	Emitted when Input is removed for this page.
`page.output_created`	`Page, Output`	Emitted when a new Output is created for this page.
`page.output_removed`	`Page, Output`	Emitted when Output is removed for this page.
`page.taxonomy_created`	`Page, Taxonomy`	Emitted when a new Taxonomy is created for this page.
`page.taxonomy_removed`	`Page, Taxonomy`	Emitted when Taxonomy is removed for this page.
`page.term_created`	`Page, Term`	Emitted when a new Term is created for this page.
`page.term_removed`	`Page, Term`	Emitted when Term is removed for this page.

Registering new signals

You can also create new signals and add them to the global scope so they can be used by the other plugins. To avoid problems with the order of plugins loading, you should access them by name with site.signals.register_signals(name). This method, when called consecutively, will always return a single instance of the signal. In fact, it is used by Stag to create built-in global signals.

Consider this example:

Emitting plugin: emitter.py

def emitting_plugin_finished(site):
    pages_no = 123
    answer_to_everything = 42
    site.signals.mysignal.emit(pages_no, answer_to_everything)


def register_plugin(site):
    site.signals.register_signal("mysignal")
    site.signals.processors_finished.connect(emitting_plugin_finished)

Subscriber plugin: subscriber.py

def print_answer(pages_no, answer):
    assert answer == 42, "Wait, what?"
    for i in range(pages_no):
        print(f"{i}: the answer to life, universe and everything is {answer}.")


def register_plugin(site):
    site.signals.register_signal("mysignal").connect(print_answer)

Note	Remember that custom signals won’t be called by Stag at any point, so you have to subscribe to any of the predefined signals first and emit your custom signals from the plugin code.

Example: Handling new file type

Suppose that we’d like to use a custom file types and generate our site from it. We have to keep in mind about the following things:

URL scheme of the files
reading input and metadata
processing input and metadata to the output

We can do all of these things in a single plugin:

from stag.ecs import Content, Metadata


def is_myft(page):
    return page.source and page.source.ext == "myft"


def is_parsed_myft(page):
    return page.input and page.input.type == "myft"


def deduce_url(path):
    if path.filebase == "index":
        return path.reldirname
    return os.path.join(path.reldirname, path.filebase)


def read(page):
    if not is_myft(page):
        return
    if page.input:  # e.g. from cache
        return

    parsed_content = []
    with open(page.source.path) as file_:
        # do reading of file here, for example like this:
        for line in fd:
            parsed_content.append(line.strip())

    page.metadata = Metadata(title="mypage", date="2021-09-10")
    page.input = Content("myft", "\n".join(parsed_content))


def generate(site):
    for page in site.pages:
        if not is_parsed_myft(page):
            continue
        if page.output:  # e.g. from cache
            return

        # convert() not covered here, because it most likely contains a
        # parser of your custom format
        html = convert(page.input.content)
        page.output = Content("html", html)


def register_plugin(site):
    site.signals.page_added.connect(read)
    signals.processors_init.connect(generate)
    site.readers.register_reader(deduce_url, lambda p: p.ext == "myft")

Using Cache

Normally Stag caches generation results for ordinary pages and plugins can (probably should) use this information to skip parts of their work and speed up site generation.

Only individual pages are cached. Plugins are thus advised to inspect page components and see if they are set. For example, if plugin is a reader, then it should check whether input entity is empty. If it isn’t, then it should skip it’s work.

Note	This behaviour has other advantage. If there are many plugins of similar type (e.g. many readers), then well-behaved plugins won’t overwrite work of their colleagues.

Example of skipping a reader

def read(page):
    if page.input:
        return
    ...


def register_plugin(site):
    site.signals.page_added.connect(read)

Plugins also have a possibility to explicitly check whether the page is read from cache by inspecting a special cached entity. This is useful for example for "mutator" kind of plugins: the ones which change already set entity. Built-in macros plugin work this way.

Example of sipping work for "mutator" plugin

def mutate(site):
    for page in site.pages:
        if page.cached:
            continue
        ...


def register_plugin(site):
    site.signals.readers_finished.connect(mutate)

Note	Plugins don’t have to worry about `--no-cache` flag.

1. Technically reading and writing could be plugins on their own, but having them outisde of the plugin system brings more benefits to the table than the counteroption