Zet

Zet is command line (CLI) notes manager. Initially it was built to address some pain points with CLI notes management, especially for Zettelkasten-like notes and knowledge bases. It is generic and isn't inherently bound to any particular methodology or notes format.

To quickly get started with Zet, see Quick Start.

Quick Start

Let's get started! In this chapter you'll get all the necessary informations to setup Zet on your machine as quickly as possible!

Installation

1. Download zet from https://git.goral.net.pl/zet.git/:

   $ git clone https://git.goral.net.pl/zet.git
   

2. Compile and install

   $ cd zet
   $ make
   $ make install
   

   make install respects PREFIX and DESTDIR environent variables. By default zet is installed to ~/.local. If you wish to install Zet system-wide, you should change make invocation to something like PREFIX=/usr make install

3. Initialize the first notebook

   $ cd ~/notebooks
   $ zet init
   

   If you already have a directory with your notes, you can point zet to it: zet init path/to/notes "My Notebook Name"

4. Set $ZET_ROOT to use zet from anywhere

   $ echo "export ZET_ROOT=\"${HOME}/notebooks\"" >> ~/.bashrc
   

   Zet can also work without setting $ZET_ROOT. In that case you'll have to enter your notebook's directory (or any child directory) and run Zet commands from there. This mode is useful if you have many notebooks scattered over the file system.

Synchronization

If you keep your notebooks in Git repositories, you may use git-sync to synchronize them. zet sync is a command which you should use each time you want to synchronize.

The prerequisite for all below methods is that git-sync script is downloaded and installed in your $PATH.

Existing Remote Repositories

If you have a remote Git repository which you'd like to automatically download when you invoke zet sync, you can configure it in .zet/zet.toml.

1. Add configuration for the new notebook to .zet/zet.toml. Point remote field to Git address you would like to fetch.

   [[notebooks]]
   name = "My Synchronized Notebook"
   path = "my-notebook"
   remote = "ssh://git@example.com/my-notebook.git"
   

2. Run zet sync

Existing Notebook

If you already have a notebook, you can synchronize it with Git simply by making sure that it points to the remote Git repository.

1. Initialize local Git repository

   $ cd notebook
   $ git init
   $ git add *.md
   $ git commit -m "Initialize notebook"
   

2. Initialize remote

   $ git remote add origin <url>
   

3. Configure git-sync

   $ git config.branch.master.sync true
   $ git config.branch.master.syncNewFiles true
   

4. Run zet sync

5. (optional, but recommended) Add remote field to notebook's section in .zet/zet.toml to easy and fast cloning in the future.

Local Only Notebooks

This is a variation of Existing Notebook method which stores notebook in local Git repository. There isn't any remote configured, so all notes stay locally on your computer. Notes are automatically committed during zet sync.

1. Initialize local Git repository

   $ cd notebook
   $ git init
   $ git add *.md
   $ git commit -m "Initialize notebook"
   

2. Run zet sync

Cheat Sheet

Write a note

$ zet note
$ zet note notebook:note.md

Setup a note's template

Note's template can be kept inside the notebook and accessed through a selector.

$ cat <<EOF | zet note --stdin template.md
Captain's Log, stardate {dt}

{input}
EOF

$ zet note -t template.md
$ echo "quick note" | zet note -t template.md --stdin

Append note

Repeatedly using the same note name will append to the file instead of replacing it.

$ echo Hello | zet note quicknotes.md --stdin
$ echo "Is it me you're looking for?" | zet note quicknotes.md --stdin
$ cat "$(zet ls quicknotes.md)"
Hello
Is it me you're looking for?

Open existing notes

$ zet open notebook:note.md

Read the notes

$ zet show notebook:note.md

Search

$ zet search "lost money"
$ zet search --or "budget.*taxes" "budget.*expenses"

Search and replace

$ zet replace "foo bar" "baz"
$ zet replace --interactive "regular.*expression" "Regular Expression"

List all available commands

$ zet commands

Working with Many Notebooks

Many notebooks can be collected under a single $ZET_ROOT. by adding a separate [[notebooks]] section for each of them inside zet.toml:

[[notebooks]]
name = "0"
path = "notebooks/first one"

[[notebooks]]
name = "1"
path = "notebooks/second one"

Default Notebook

You can query and set a default notebook with zet default-notebook. If default notebook was never set, the first notebook in zet.toml will be used as a default.

Default notebooks are important due to their role in filling the notebook gap in selectors. Whenever you don't specify a notebook in selector, the default one will be used.

Core Commands

Core commands are the ones built into Zet. You can't change them or remove them. They are the ones which are listed when running zet --help.

zet init

zet init initializes a new notebooks collection in a current directory. It creates initial .zet/zet.toml configuration file.

Without arguments, zet init will create a new notebook and initialize zet.toml with data about it. You can change the defaults by passing to zet init a path and a name.

zet commands

zet commands prints all installed subcommands, together with a short description of each one (if available).

zet api

Run plumbing, easy to parse API commands which provide other subcommands a common interface to access and compute Zet implementation details. See the chapter about API.

zet env

Print environment which will be passed to subcommands. It is used mostly for debugging purposes.

Non-core Commands

Non-core commands are all the commands which aren't built into Zet. Zet discovers them dynamically by inspecting $PATH and $ZET_MODULES_PATH. This page lists essential non-core commands distributed with Zet. You can also read their respective help pages by running zet <cmd> --help. You can list all non-core commands available on your system by running zet commands.

zet default-notebook

This command prints current default notebook. You can also change default notebook by running zet default-notebook <notebook-name>.

<notebook-name> is the name of notebook configured in zet.toml, not its path. You can list all currently configured notebooks by running zet notebooks.

zet gen

Generate a file name from a format string. The following format tokens are available:

• %u - new UUID¹ (requires uuidgen program installed)
• %su - short UUID (8 first characters of UUID; requires uuidgen program installed)
• %d - current date and time, with any non-numeric characters being stripped
• %r - random number

Examples

$ zet gen file-%su.txt
file-f391b9a0.txt

zet import

Import files and URLs into the notebook. Imported files are automatically copied to the default notebook. URLs are automatically downloaded and converted by Pandoc into Markdown files.

You can change the default destination of imported files with -d switch.

zet import never overwrites files. If file with a chosen destination already exists, zet import will enumerate it.

Examples

$ zet import https://example.com
https://example.com imported to /home/user/notebooks/0/url.md

$ zet import https://example.com
https://example.com imported to /home/user/notebooks/0/url-1.md

$ zet import ~/some/document.pdf
/home/user/file.pdf imported to /home/user/notebooks/0/document.pdf

$ zet import ~/some/file.pdf -d 0:imported.pdf
/home/user/file.pdf imported to /home/user/notebooks/0/imported.pdf

zet ls

List files inside notebooks. By default all files from all notebooks are listed. This selection can be narrowed down by passing one or more selectors. If any selector expands to a directory, all files in that directory are listed, recursively.

Along file paths, zet ls can also print note titles of known formats by inspecting file contents.

Examples

$ zet ls 0:
0/note.md
0/other-note.md


$ zet ls 0: -t
Note Title::0/note.md
::0/other-note.md

$ zet ls 0:note.md -t -F,
Note Title,0/note.md

zet mv

Rename a file. This command accepts either the source and destination selectors, or source selector and --to-title switch, which renames a file to the canonical title.

Canonical title is a note's title converted to the form which is suitable for file names. Typically it is lower-cased, has stripped whitespace and some other special characters.

Renaming can be performed interactively by passing -i switch.

Examples

$ zet mv 0:note.md 0:newname.md
$ zet mv 0:newname.md --to-title -i

zet note

Write new notes in a format preferred by the default_filetype option. Normally zet note opens editor, but it's also possible to add new notes by piping them into zet note --stdin. Notes can be put into templates, which is especially useful for notes piped into zet note.

By default zet note will place notes in newly created files named after detected note's title. Alternatively you can provide a selector with note's destination. If it exists, zet note will append to that file.

You can change the default editor by passing it to --editor switch. It accepts a format string with {} placeholder, which will be replaced with a file path to the temporary note file.

Templates

Templates are ordinary files and can be stored inside a notebook. They're chosen with -t switch which accepts a selector. Template files may contain the following placeholders, which will be automatically substituted when creating the note:

• {dt} - current date/time
• {host} - hostame of the current system, as reported my hostname program
• {user} - name of the current user
• {input} - note piped into zet note --stdin

Examples

$ zet note

$ zet note 0:quicknotes.md

$ echo "Hello" | zet note --stdin -t 0:note-template.md

$ zet note --editor='my-editor {}'

zet notebooks

List the notebooks, optionally formatted. Format string passed to -f switch accepts the following tokens:

• {name} - configured notebook's name
• {path} - notebook's path, relative to $ZET_ROOT
• {abspath} - notebook's absolute path

zet open

Open notes and files from a notebook. This command accepts a selector with a twist: it might expand to a partial file path, which is examined against all the files. If there is a match, it is opened with the opener program (e.g. xdg-open). If there are many matches, zet open interactively queries which one should it open.

Only the file part of selector is matched against the files.

Default opener can be changed with -p switch. It accepts {} placeholder which will be replaced with actual file path to open.

Examples

$ zet open 0:ot
More than one file matches. Which one to open?
[1] 0/other.md
[2] 0/robots-take-over-the-world.md

zet replace

Finds all occurences of a chosen regular expression and replaces them in-place with a string. Search is performed in all files in all notebooks. Alternatively, selectors can be passed as a third and further arguments to narrow the list of files in which search-replace occurs.

Search can be case-insensitive (-I) and interactive (-i). In interactive mode, zet replace will print a diff with proposed changes in each file, which must be accepted before doing the actual replacement.

Examples

$ zet replace 'some.*text' "other text"
Search-replacing through 14 files...

$ zet replace 'some.*text' "other text" -i
Search-replacing through 14 files...
--- Current version of 0/note.md
+++ After replacement of 0/note.md
@@ -1,2 +1,2 @@
-This is some of my text
+This is other text

>> Accept changes in 0/note.md? ([Y]es / [N]o / [Q]uit)

zet search

Search the notes for selected queries. Each query is a phrase or regular expression which is expected to match. When more than one query is given, all of them must be present in a file to return a match. --or switch can be used in cases when any of passed queries must be present to return a match.

Search scope can be narrowed down by passing selectors through -s switch. Searches can be performed case-insensitive with -i switch.

zet search uses grep programs found on the system to perform the actual search and presents their output in the unified way. zet search will try to select the fastest available program, but you can enforce one with --grep switch.

zet search will truncate many long matches. You can control this behaviour with --show-all switch.

Examples

$ zet search a e
0/poetry.md:
    I am here
    You are there
    They are everywhere
    This is not a very great poem
    But hopefully someone stops me at some point
(... result truncated, run 'zet search --show-all' to show all matched entries ...)

0/butcher.md:
    Aaaah, fresh meat

$ zet search a y
0/poetry.md:
    They are everywhere
    This is not a very great poem
    But hopefully someone stops me at some point

$ zet search a -l
0/poetry.md
0/butcher.md

$ zet search one two --or
0/ones.md
    one
    one
    certainly not two
    and not the other one

0/twos.md
    two
    two

$ zet search 'regex.*engine' -l -s nb1: -s nb2:
nb1/note.md
nb2/other-note.md

zet show

Show contents of a note inside the terminal, automatically choosing the fanciest formatting program, like less or batcat. You can force using any particular program with -p.

In similar fashion to zet open, zet show accepts a partial selector which interactively queries which file should be opened if partial selector matches more than one.

Implementation detail: zet show is actually only a wrapper for zet open.

Examples

$ zet show glados.md
───────┬────────────────────────────────────────────────────────────────────
       │ File: /home/user/notebooks/0/glados.md
───────┼────────────────────────────────────────────────────────────────────
   1   │ # Glados Anthem
   2   │
   3   │ This was a triumph!
   4   │ I'm making a note here: "HUGE SUCCESS".
   5   │ It's hard to overstate my satisfaction.

zet sync

Automatically synchronize notes in notebooks which are git repositories. It requires git-sync to work and works faster for many notebooks if GNU Parallel is present on the system.

Please refer to Synchronization chapter for details.

zet tasks

Show tasks in notebooks. Tasks are lines with checkboxes in form of - [ ] for open and - [X] for closed tasks.

Examples

$ zet tasks
0/todo.md
---------
- [ ] task 1
    - [ ] subtask

$ zet tasks -s closed
0/todo.md
---------
- [X] finished task

Selectors

A lot of commands which work on files use selectors syntax in place where you'd normally put a path to the file. Aim of selectors is to simplify the way these paths are provided.

Zet expands selectors to ordinary paths. From the left, before the colon, selectors have notebook's name (not path), separated by the colon and then relative path of a note inside that notebook. Each of these parts is optional.

Selectors look like this:

selector := [notebook:][directory/][note.md]

For example notebook:note.md, note.md, subdir/ and notebook:subdir/other-note.md are all selectors.

Each part of selector is optional. Selectors can expand both to files and to directories. Paths to which selectors expand don't have to exist at all. In some situations it is then important to be aware of the rules which apply to selector expansion.

Selector Expansion Rules

• If selector is an absolute path, it is left untouched;
• if notebook part is missing, selector expands with a default notebook in place of it;
• if path part (i.e. directory/note.md) is missing, selector expands to the path to notebook's directory;
• if selector expands to the path which doesn't exist on a file system:
  • if selector ends with a slash character (/ or \\), then it is treated as a directory;
  • if selector doesn't end with a slash character, then it is treated as a file.

You can test selectors with 2 API commands: zet api paths <selector> and zet api is-file <selector>

Examples

Let's suppose that ZET_ROOT=$HOME/notebooks and that we have following file structure:

~
└── notebooks/
    ├── .zet/
    │   └── zet.toml
    ├── notebook1/
    │   ├── subdir/
    │   │   └── note.md
    │   └── note.md
    ├── notebook2/
    │   └── note.md

We name our notebooks: "1" for notebook1 and "2" for notebook2. Default notebook is notebook1. In that case, we'll have the following selectors:

Examples in Commands

• zet ls 1: will list all files in a notebook "1"
• zet ls 1 lacks a colon after "1", so it will list a single file "1" in a notebook "1" (equivalent of zet ls 1:1)
• zet mv 1 2/ will move a file named "1" from a notebook 1 to a directory named "2" in a notebook 1
• zet note foo/ will create a new note in a directory "foo"
• zet note foo will create a note "foo" in a default notebook. If it already exists, zet note will append to it

Hooks

Hooks are programs you can place in a hooks directory to trigger actions at certain points in zet's execution.

Hooks directory is $ZET_DIR/hooks (.zet/hooks). Zet searches it for executables which are invoked when they're search accordingly. Hooks without executable bits set are ignored.

Zet passes informations about the event which triggered hook via command line arguments. They're documented below. Hooks also receive the same environment as ordinary Zet commands. Notably, $ZET_ROOT and notebook variables parsed from zet.toml are set.

Zet doesn't change a current working directory before invoking hooks, but respects --directory switch.

Hooks can be disabled by using --no-hooks switch or by setting ZET_NO_HOOKS=1 environment variable. Setting the environment variable inside the hook has effect of disabling nesting of hooks for all consecutive Zet commands.

Hook Types

pre-command

This hook is invoked before any non-core Zet subcommand is run, such as zet mv, zet note or zet sync. Core subcommands, such as zet init, zet env or zet api do not trigger this hook.

If pre-command fails by returning a non-zero exit code, it will prevent execution of Zet command itself.

Zet passes the following arguments to it:

• subcommand which triggered the hook, without the zet- prefix, for example mv, note or sync
• all arguments which are passed to the subcommand

Arguments are passed exactly as they were passed to Zet (so they might be expanded by the underlying shell, but notably selectors will not be expanded by Zet).

post-command

This hook is invoked after Zet subcommand finishes. It receives a flag, which takes a value of 0 or 1 and denotes whether Zet command was successful or not. 0 means success. Other parameters are the same as in pre-command hook.

Zet won't trigger post-command hook when it was unable to run a command (for example when user invoked a subcommand which doesn't existy).

Configuration

Zet reads the configuration inside TOML's .zet/zet.toml file. Empty zet.toml is created by running zet init.

Configuration Example

[[notebooks]]
name = "main"
path = "content/0"
default_filetype = "adoc"
remote = "ssh://git@example.com/notebook.git"

[[notebooks]]
name = "journal"
path = "content/my journal"
default_filetype = "org"
remote = "ssh://git@example.com/journal.git"

Notebooks Array

Zet discovers all of managed notebooks by reading [[notebooks]] array entries. Order in which [[notebooks]] are declared in zet.toml matters for evaluating the default notebook.

It has the following keys:

name

(mandatory)

Name of the notebook which will be used in selectors to access it. It can be any string, but for ease of access it's preferabbly a short single word, all lower-case.

path

(mandatory)

Relative path to the root directory of notebook. path is relative to the $ZET_ROOT, i.e. to the .zet directory.

default_filetype

(optional)

Change the filetype for notes which Zet should prefer when it creates new ones (for example with a bare zet note). If this option isn't set, Zet will default to md.

default_filetype should be a file extension without the initial dot.

remote

(optional)

Git URL used by zet sync to automatically initialize the notebook from remote repository. See the chapter about synchronization.

Custom Commands

Core vs Non-core

There are 2 types of zet subcommands: core and non-core. Core commands are built into zet. They cannot be changed or replaced. Their list is available by running zet --help. Some examples of core commands are:

• zet init
• zet api
• zet env
• zet commands

Non-core commands are all the other commands. This includes some commands which are bundled with zet. These subcommands can be removed or replaced. Some examples are:

• zet ls
• zet mv
• zet note
• zet open
• zet sync

Search Paths

When you run a non-core zet command, zet searches for it inside the following directories: ${ZET_MODULES_PATH}:${PATH}. If $ZET_MODULES_PATH environment variable is not defined, zet defines one as a set of local-specific data directory and system-specific data directory:

• (Linux) ${XDG_DATA_HOME:-$HOME/.local/share}/zet/modules
• (macOS) $HOME/Library/Application Support/zet/modules
• (Windows) {FOLDERID_RoamingAppData}/zet/modules
• (Linux, macOS) /usr/share/zet/modules

You can see the values of $PATH and $ZET_MODULES_PATH by running zet env.

If you redefine $ZET_MODULES_PATH, zet might loose the possibility to run bundled non-core subcommands. If it isn't what you intended, first check the default $ZET_MODULES_PATH calculated by zet and use it:

export ZET_MODULES_PATH=/my/dir1:/my/dir2:$HOME/.local/share/zet/modules:/usr/share/zet/modules

All executables found in these directories which start with zet- prefix are considered zet subcommands and will be listed on zet commands screen.

Writing own subcommands

To add your own subcommand, simply drop an executable (for example a shell script) in your $ZET_MODULES_PATH or $PATH. You should prefer $ZET_MODULES_PATH to not clobber the $PATH and to avoid the risk of accidentally running subcommands outside of zet's execution environment.

Environment

When Zet calls any subcommand, it parses zet.toml and exports all variables in each [[notebooks]] section into subcommand's environment. Variables are named after notebook's name and prefixed with ZET_NOTEBOOK_. For example, if there exists a notebook named "My Notebook", there will be at least ZET_NOTEBOOK_MY_NOTEBOOK_NAME and ZET_NOTEBOOK_MY_NOTEBOOK_PATH exported.

List of notebook names will be exported as ZET_NOTEBOOKS colon-separated list.

You can inspect all Zet-specific environment variables by running zet env.

API

To ease the task of writing your own subcommands, zet provides the api core command, which you should use for some common tasks. See zet api --help and zet api <api-command> --help for in-depth description of what's availale.

For example, your subcommand might need an access to the files using the selector syntax, defaulting to all the files inside all the notebooks. You can use zet api list and zet api notebooks to access all relevant informations:

#!/bin/bash
# usage: zet command [selectors...]

if (( ! $# )); then
    # readarray handles possible spaces inside the paths returned by api calls
    readarray -t args <<< $(zet api notebooks --as-selectors)
    set -- "${args[@]}"
fi

readarray -t paths <<< $(zet api list "$@")

for p in "${paths[@]}"; do
    echo "$p"
done

Libraries

Zet exposes libraries for some languages inside $ZET_LIB_DIR directory. These libraries are language-specific and contain helper functions which are useful for commands written in a particular language. Libraries are stored in separate directories, e.g. Bash library is in $ZET_LIB_DIR/bash and Python library is in $ZET_LIB_DIR/python.

Descriptions

zet commands lists all found subcommands. Each subcommand may additionally have a short, one-line description which will be shown to user by zet commands. These descriptions are gathered from command-list.txt files found in $ZET_MODULES_PATH directories. Typically, each command-list.txt should have descriptions of all zet commands in the same directory.

Only directories in $ZET_MODULES_PATH (and not in $PATH) are used to gather command-list.txt files. It means that if you use $PATH to maintain your collection of submodules, you won't be able to set their descriptions.

Format of each line in these files is:

subcommand: description

Example

Suppose that ZET_MODULES_PATH=$HOME/zetmodules and that inside this directory there are 2 subcommands: zet-foo and zet-bar. To add descriptions for them, have to create $HOME/zetmodules/command-list.txt file with the following contents:

foo: foo your notes by fooing them all at once
bar: make barring sound from your notes

Subcommand Templates

Bash

#!/bin/bash

#/ usage: zet mycommand [options] <selector>
#/
#/ Foo a bar
#/
#/ Arguments:
#/   selector
#/          foos a bar without blahing
#/
#/
#/ Options:
#/   --help
#/          show this help message

set -o errexit
set -o nounset
set -o pipefail

. "${ZET_LIB_DIR}/bash/zetlib.bash"

opt_short="h"
opt_long=("help")
parse_opts "$@"
set -- "${OPTRET[@]}"; unset OPTRET

while (( $# )); do
    case $1 in
        -h|--help)              usage ; exit 0 ;;
        --)                     posargs=() ;;
        *)                      posargs+=("$1") ;;
    esac
    shift
done

zet api paths "${posargs[@]}"

Python

#!/usr/bin/env python3

import os
import sys
import argparse

sys.path.append(os.path.join(os.environ["ZET_LIB_DIR"], "python"))
from pyzet import api_paths


def prepare_args():
    parser = argparse.ArgumentParser(description="Foo a bar")
    parser.add_argument("selector", help="foos a bar without blahing")
    return parser.parse_args()


def main():
    args = prepare_args()
    lines = api_paths(args.selector)
    print(*lines)


try:
    sys.exit(main())
except KeyboardInterrupt:
    sys.exit(15)