Browse Source

add make and pandoc to issue 2

pull/6/head
Ben Harris 2 years ago
parent
commit
fd85af8e37
  1. 136
      issues/2/make.md
  2. 135
      issues/2/pandoc.md

136
issues/2/make.md

@ -0,0 +1,136 @@
# makefile magic
author: [ben](https://tilde.team/~ben/)
i've used makefiles plenty of times for a variety of projects, but my
understanding has not really improved beyond the most basic level.
working with pandoc for the [\~club wiki](https://tilde.club/wiki/) and
the [zine](https://zine.tildeverse.org/) inspired me to replace a [janky
build
script](https://github.com/tildeclub/site/blob/25e3ed375791d48d085b434beabac10f584f5540/wiki/build-wiki.sh)
with some makefile goodness.
feeling a loss for where to start, i reached out to
[tomasino](gopher://tilde.black) for some guidance. he responded that i
should have a look at a makefile he built for a javascript project
template which he linked on [his
github](https://github.com/jamestomasino/ts-boilerplate/blob/master/Makefile).
let's go through some of my biggest takeaways from inspecting tomasino's
example and creating several of my own makefiles from it.
## anatomy of a makefile
this was something that i thought i understood, but realized i didn't
fully grok.
i *think* that i understand it now, or at least know now what to search
for when i get stuck the next time.
let's start with the basics: a makefile should be named Makefile (note
the capital M). the basic syntax consists of a target followed by a
colon (`:`) and an optional list of requirements, then a list of
commands to be run (indented by literal tabs).
myproc: source.c
cc -o myproc source.c
this example defines how to build myproc, which depends on source.c.
### variables
makefiles use variables in two main formats.
- recursively-expanded: evaluated every time they're referenced, even
in other variables
- defined with a single `=`
- simply-expanded: evaluated once at the time of definition
- defined with `:=`
simply-expanded variables are generally more predictable and will likely
be what you should use in most cases.
x := foo
y := $(x) bar
x := later
will become
y := foo bar
x := later
variable names are case-sensitive. you'll likely see all-caps variable
names more often than not.
### guided example
this was the major revelation for me coming from my brittle buildscript
that deleted and recreated everything on each run.
make wants to know what files you want built, as well as how to build
them.
let's take the \~club wiki as an example: we want to build one html for
each markdown file in the source directory. how do we tell make which
files to build and how to build each one?
using a standard static target name, you'd need to add a target for each
markdown file.
ssh.html: source/ssh.md
pandoc -so ssh.html source/ssh.md
git.html: source/git.md
pandoc -so git.html source/git.md
this already feels like a lot of duplication. let's stay
[DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself). here's an
example from tomasino's makefile that i adapted for the wiki: [github
source](https://github.com/jamestomasino/ts-boilerplate/blob/master/Makefile#L46)
$(DST_DIR)/js/%.js: $(SRC_DIR)/ts/%.ts
$(mkdir)
$(tsc) $< --outFile $@
in this example, we're telling make that files under `DST_DIR` are built
from `SRC_DIR` with a command defined in the `tsc` variable.
let's adapt this to our wiki example:
%.html: source/%.md
pandoc -so $@ $<
here we're telling make that it can build html files from the
corresponding md file in source by running pandoc. but how do we tell
make which html files we want to build? let's go back to tomasino's
[example](https://github.com/jamestomasino/ts-boilerplate/blob/master/Makefile#L18-L19)
SRC_TS_FILES != find $(SRC_DIR)/ts -name '*.ts'
DST_JS_FILES := $(SRC_TS_FILES:$(SRC_DIR)/ts/%.ts=$(DST_DIR)/js/%.js)
in this case, we're using the special assignment form with `!=` that
uses the output of a shell executable. we now have a list all the `.ts`
files that need to be compiled in `SRC_TS_FILES`. next, we assign
`DST_JS_FILES` with the special replacement syntax, which changes
`ts/%.ts` into `js/%.js`. so, these variables now contain a list of
source and desired files, along with a definition of how to build those
files.
here's how i adapted this step for the wiki:
SRC_MD_FILES != find source -name '*.md'
DST_HTML_FILES := $(SRC_MD_FILES:source/%.md=%.html)
this means that make now has a list of html files that are required
based on the list of all markdown files in the source directory.
now all that's left is to tell make that we want to build all of the
`DST_HTML_FILES`. let's add that list to the `all` target:
all: $(DST_HTML_FILES)
now we can generate all html files by simply calling `make`
here's the completed
[Makefile](https://github.com/tildeclub/site/blob/master/wiki/Makefile).

135
issues/2/pandoc.md

@ -0,0 +1,135 @@
# pandoc magic
author: [ben](https://tilde.team/~ben/)
[pandoc](https://pandoc.org) is an incredibly powerful tool for creating
and converting documents.
i recently started using it for all kinds of different things,
including:
- the [tilde.club wiki](https://tilde.club/wiki/)
- my personal [page on \~club](https://tilde.club/~benharri/) (source
[here](https://tildegit.org/ben/club-site))
- this zine itself (see [zine
makefile](https://tildegit.org/tildeverse/zine/src/branch/master/Makefile))
let's look at some of the tricks and tips that i've learned from these.
## format markdown files
this seems like it might not work, but `pandoc` will format your
markdown files for you if you convert the source file from markdown *to*
markdown again. for example, if you wanted to tidy up your `README.md`,
you could run:
> `pandoc -f markdown -t markdown -o README.md README.md`
note that if you want to preserve any yaml frontmatter blocks, you'll
need to change the from and to types to `markdown+yaml_metadata_block`.
another thing to note is that you can preserve github-formatted markdown
by using `gfm` instead of `markdown`. additionally, if you prefer to use
atx-style headers, don't forget to add the `--atx-headers` switch.
a complete example:
pandoc \
-f markdown+yaml_metadata_block \
-t markdown+yaml_metadata_block \
--atx-headers \
-o README.md \
README.md
## templates
pandoc ships with a full set of templates that are used to control how
things are displayed when writing to certain formats.
let's look at the templates for html and latex. to get the default
template for a given format, use `-D`. the template is written to
stdout, so let's save it to a file.
> `pandoc -D html > html.template`
open up this template in your editor and you'll see what's used when you
convert things to html.
important note: these templates are only used when generating standalone
documents (the `-s` or `--standalone` flag).
note that the template uses lots of variable substitutions. we can set
values for these in yaml frontmatter or in the command invocation.
to make some basic customizations, we can fill in the metadata values
that will be automatically replaced in the document when we build.
however, some circumstances require a custom layout or additional
changes to meet the requirements.
have a look at the
[wiki.tmpl](https://github.com/tildeclub/site/blob/master/wiki/wiki.tmpl)
used on the [\~club wiki](https://tilde.club/wiki/). some of the notable
changes include:
- table of contents title
- hardcoded stylesheet
- link to author's user page
the stylesheet change would be possible on the command line by using the
`-c` or `--css` option, but the other changes require a custom template.
try it out on your own project!
## lua filters
sometimes the rendered page still isn't what you're looking for. maybe
you need some additional tweaks (css classes, extra html items, etc).
pandoc has supported filters for a long time in the form of json passed
around on pipes that allows you to modify the internal data structures
before they're written to the output format.
the main downside of using these filters is that it introduces another
layer of dependencies (namely the language that the filter's written in
and also the library for that language to interact with pandoc's json).
as of pandoc 2.0, a lua interpreter with a class library for creating
filters is built in to the pandoc executable. let's go over some basic
examples.
### permalinks for headers
a common feature on most html created via markdown is a small link
displayed next to the header so that you can deep-link directly to that
section of the page. this is frequently seen on github and other
documentation sites.
here's a very simple way to create the link:
function Header(elem)
table.insert(elem.content, pandoc.Space())
table.insert(elem.content, pandoc.Link("§", "#" .. elem.identifier))
return elem
end
save this to a file and call it in your pandoc conversion with the
`--lua-filter` option.
### change header levels
this is an example from this zine. i wanted to decrease the size of
headers below h1 level without changing the external css. here's how i
did it.
function Header(elem)
if elem.level > 1 then
elem.level = elem.level + 1
end
return elem
end
this changes all header elements to be one level smaller.
there are additional examples on
[pandoc.org](https://pandoc.org/lua-filters.html) if you'd like to see
more.
Loading…
Cancel
Save