slope

slope is an s-expression based language loosely modeled after scheme R4RS. It does, however, diverge quite a bit in some areas (especially IO and filesystem related constructs). slope also does not include many standard constructs of scheme (such as let, *let, case, etc.) in favor of using other existing constructs (this often involves slightly longer code that uses a narrower range of generalized constructs).

The interpreter is implemented in golang and many of the library differences between slope and scheme are based around generalizing the go standard library into a usable base system for a dynamically typed language. A great example of this generalization is that in slope you can use the function write to easily write to a file (including stdout/err), a tcp connection, or a buffer object. slope knows what you are trying to do based on what you pass it, so separate file-write, tcp-conn-write, string-buf-write functions are not necessary. Even with a runtime panic, slope will also attempt (I will stop short of guarantee) to close any open files or net connections. Other niceties are planned.

Why?

Like many of my projects: to learn and have fun. This interpreter is based on some existing code. I rewrote sections of the lexer and parser (areas I enjoy working in), left the apply and environment setup alone, and made a few small changes to the eval function. As such, this could be considered a fork of that code with some added core language constructs as well as a significantly expanded library (another area I enjoy working in). I have definitely learned different things than my previous language project while working on this codebase, and I think have already (early days as I am writing this) ended up with a much more realistically usable language (though that is certainly contextual).

What is implemented?

As mentioned above, this is not a scheme implementation in any proper sense and it will go its own way when it sees fit.

Types

slope recognizes a few types, that are generally created and used dynamically:

• number
  • implemented as float64 under the hood
  • hexidecimal can be written with a leading 0x, as in 0xFFF
  • octal can be written with a leading 0, as in 077
  • in strings, values can be escaped to characters as decimal, octal, or hex: "\27[1mI am bold text\033[0m, I am not bold. I am a hex based char: \0x84"
• symbol
  • strings that represent language objects/constructs/vars
  • can be created using (quote my-symbol) or 'my-symbol
• bool
  • as #t and #f
  • anything can be cast to bool with a #t value, except #f (the only truly falsy value without using ~bool, see below)
• string
  • anything in "quotes"
  • allows for backslash escapes within the quotes (for chars by number or shortcuts to tab, newline, etc)
• list/pair
  • includes the empty list (null)
  • lists are the primary data structure
• exception
  • an error response
  • can be created using (! "My exception text")
  • can be tested for with (exception? some-var)
  • two functions exist to set behavior for exceptions:
    • (exception-mode-panic), will panic and halt execution when an exception is encountered (default)
    • (exception-mode-pass), will return the exception like any other value and it can be handled
    • the two modes can be switched on and off at various points in the code

There is also the concept/type IOHandle. This type cannot be cast to another (except string, but the output will likely not be useful for much beyond debugging), nor can it be entered manually. It is returned by things like net-conn and file-open-read. It is the general concept of something that can be read from and/or written to.

Special Forms

There are a number of special forms in the language that will allow, for example, un-initialized symbols to be passed (for example, in define), among other things less available from regular lambdas (which are themselves a special form).

• quote
  • Can be shorthanded so that '(1 2 3) is equal to (quote (1 2 3))
• if
• and
• or
• cond
• set!
• define
  • Lexically scoped, such that its use from within a lambda scopes any defined variables to the scope of that lambda and its child scopes
• lambda
  • The body of a lambda has an implied (begin ...) surrounding it. As such, you may place non-nested procedure calls or values inside of a lambda body and the last will be returned
  • Variadic lambdas can be created by using the special param name args-list (which will be a list of all of the remaining arguments when accessed rom the lambda body). This param should be the last param in the lambda definition, as it will eat all arguments that come after its position. Any params defined after args-list have undefined behavior (they will generally not be created)
• begin
• begin0
  • Like begin, but returns the value resulting from the evaluation of the first expression. No shorthand exists for begin0
• filter
  • Takes a procedure that, in turn, takes a single argument. Each value from the second argument, a list, will be passed to the procedure. Anything truthy results are returned in a new list
• load
  • Loads and executes a slope file. Absolute path to file must be used
• load-mod
  • Loads a module. Module name should be used
• load-mod-file
  • Used within modules to load additional files
• usage
  • Used to display the procedure signature and any additional information about the procedure, mostly used interactively
• eval
  • Executes representations of slope code

While not a special form from a compiler implementation standpoint list has some syntactic sugar that is worthy of note. These four lines will all produce the same list:

(list 1 2 3)
[1 2 3]
(quote 1 2 3)
'(1 2 3)

Quote and list both have some syntactic sugar to create a shorthand for their usage ('() for quote and [ ... ] for lists).

Special Forms API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (quote [expression...]): bool
• (if [condition check: expression] [true branch: expression] [[false branch: expression]]): value
• (and [expression...]): bool
• (or [expression...]): bool
• (cond [([condition check: expression] [expression])...]): value
• (set! [variable name: symbol] [value|expression]): value/expression
• (define [variable name: symbol] [value|expression|procedure]): value/expression
• (lambda [([[argument: symbol...]])] [[expression...]]): procedure
• (begin [expression...]): value
• (begin0 [expression...]): value
• (filter [test: procedure] [list]): list
• (load [filepath: string...]): symbol
• (load-mod [filepath: string...]): symbol
• (load-mod-file [filepath: string...]): symbol
• (usage [[symbol|string]]): () Will display usage information for the given procedure or runtime value. If no argument is given usage will list known procedures
• (eval [expression|value] [[evaluate-string-as-code: bool]]): value

Library

Values

PI, E, PHI, sys-args, stdin, stdout, stderr, devnull

sys-args is populated with the arguments run at the command line, similar to C's argv

stdin, stdout, stderr, and devnull are set at runtime to an open IOHandle representing each IO file. This makes it easy to use them the same as you would any other file (by passing the IOHandle to the reader/writer).

Number related

Implemented:

positive?, negative?, zero?, abs, floor, ceil, round, +, -, *, /, min, max, number->string, rand

Not implemented, but on my radar:

modulo, remainder, quotient, expt, sin, cos, tan, atan, sqrt

Number API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (positive? [number]): bool
• (negative? [number]): bool
• (zero? [number]): bool
• (abs [number]): number
• (floor [number]): number
• (ceil [number]): number
• (round [number] [[number: decimal places to round to]]): number
• (+ [number...]): number
• (- [number...]): number
• (* [number...]): number
• (/ [number...]): number
• (min [number...]): number
• (max [number...]): number
• (number->string [number] [[base: number]]): string
• (rand [[max]] [[min]]): number

Conditional

Implemented:

>, >=, <, <=, not, equal?, number?, string?, bool?, symbol?, pair?, null?, list?, procedure?, atom?, ~bool, assoc?, string-buf?, io-handle?

~bool is non-standard and will cast any value as a boolean value using a looser set of rules. Under normal conditional rules in slope anythong other than #f is considered truthy (#t). When using ~bool the number 0 is false, the list empty list () is false, and empty string "" is false, and a closed IOHandle is false. Everything else, except the literal value #f, is true (#t).

So...

> (if (~bool 0) "Truthy" "Falsy")
#1=> Falsy
> (if 0 "Truthy" "Falsy")
#2=> Truthy

Conditional API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (> [number] [number]): bool
• (>= [number] [number]): bool
• (< [number] [number]): bool
• (<= [number] [number]): bool
• (not [value]): bool
• (equal? [value] [value]): bool
• (number? [value]): bool
• (string? [value]): bool
• (bool? [value]): bool
• (symbol? [value]): bool
• (pair? [value]): bool
• (list? [value]): bool
• (null? [value]): bool
• (procedure? [value]): bool
• (atom? [value]): bool
• (assoc? [value]): bool
• (io-handle? [value]): bool
• (string-buf? [value]): bool
• (~bool [value]): bool

List related

Implemented:

length, cons, car, cdr, append, list, map, for-each, list->string, list-ref, list-sort, reverse, assoc, member?, list-seed

• list can be shorthanded by using square brackets: [1 2 3] vs (list 1 2 3)
• length and reverse will take a string or a list.
• list-sort will sort a list in ascending order by casting all elements within the list as strings and doing a compare. The exception to this is numbers, which will be compared among themselves as numbers. In a mixed list numbers will come after other values. To sort descending, pass the results of list-sort to reverse

List API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (length [list|string|exception|IOHandle:string-buf]): number
• (list [value...]): list
• (cons [value] [list]): list
• (car [list]): list
• (cdr [list]): list
• (append [list] [value...]): list
• (map [procedure] [list...]): list
• (for-each [procedure] [list...]): list
• (list->string [list] [join-on: string]): string
• (list-ref [list] [index: number] [[set: value]]): value|list
• (list-sort [list] [[sub-list-index: number]]): list
• (reverse [list|string]): list
• (assoc [assoc-list] [key: value] [[value: value]]): value
• (member? [list] [value]): bool
• (list-seed [length: number] [value]): list More efficient than recursion for building large lists and wont run into stack overflows

String related

Implemented:

length, substring, string-append, string->number, string-format, string->list, string-trim-space, string-index-of, string-ref, string-upper, string-lower, string-make-buf, string-buf-clear, regex-match?, regex-find, regex-replace, string-fields, string->md5, string->sha256

• length will take a string, an IOHandle representing a string-buf, or a list. In the case of a string it will return the number of runes as thought of in golang. Loosely equivalent to characters, rather than bytes.
• Standard read and write procedures work with the IOHandle returned from string-make-buf
• length and reverse will take a string or a list.
• string-format works like a simplified sprintf. All values are represented by %v (for value). You can pad values with spaces on the right by adding the desired field width after the percent: %10v with the string "hello" would produce " hello" and you can left justify the text with a - after the percent: %-10v for the same variable would produce "hello ". There are no other modifiers.

String API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (length [list|string|exception|IOHandle:string-buf]): number
• (substring [string] [number] [number]): string
• (string-append [value...]): string
• (string-format [format: string] [value...]): string (replaces `{}`'s in string with successive values)
• (string->number [string] [[base: number]]): number/#f If a non-decimal base is supplied, the input string will be parsed as an integer and any flaoting point value will be floored. A valid base passed with a string that does not parse to a number will return #f
• (string->list [string] [[split-point: value]] [[count: number]]): list (splits the first string at each instance of value, cast as a string)
• (string->md5 [string]): string
• (string->sha256 [string]): string
• (string-fields [string]): list
• (string-trim-space [string]): string
• (string-index-of [string] [string]): number (`-1` if the string could not be found)
• (string-ref [string] [number]): string
• (string-upper [string]): string
• (string-lower [string]): string
• (string-make-buf): IOHandle:string-buf
• (string-buf-clear [IOHandle:string-buf]): ()
• (regex-match? [string (pattern)] [string]): bool
• (regex-find [string (pattern)] [string]): list
• (regex-replace [string (pattern)] [string]): string

IO Related

Implemented:

newline, display, display-lines, write, write-raw, read-line, read-char, read-all, read-all-lines, close file-create, file-create-temp, file-open-read, file-open-write, file-append-to, file-stat, file-name

Not implemented, but on my radar:

file-seek

IO API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (newline): ()
• (display [value...]): ()
• (display-lines [value...]): ()
• (write [string] [[IOHandle]]): ()|IOHandle
• (write-raw [string] [[IOHandle]]): ()|IOHandle
• (read-all [[IOHandle]]): string
• (read-line [[IOHandle]]): string|symbol:EOF
• (read-char [[IOHandle]]): string
• (read-all-lines [[IOHandle]]): list
• (close [IOHandle]): ()
• (file-create [filepath: string]): IOHandle
• (file-create-temp [filename-pattern: string]): IOHandle
• (file-open-read [filepath: string]): IOHandle
• (file-open-write [filepath: string]): IOHandle
• (file-append-to [filepath: string] [string...]): ()
• (file-name [IOHandle: file]): string
• (file-stat [filepath: string]): assoc-list|#f

  Will return #f if the file does not exist, but the procedure otherwise runs successfully. Any other errors accessing a file will raise an exception.

  The associative list will have the following keys:

  • name
  • size
  • mode
  • mod-time
  • is-dir?
  • is-symlink?
  • path

  mode will be in decimal as a number and can be converted to an octal string with number->string. size is in bytes. path will be an absolute path after following a symlink, or the path as provided to stat if the file is not a symlink.

System Related

Implemented:

exit, license, apply, !, exception-mode-panic, exception-mode-pass, term-size, term-restore, term-char-mode, term-raw-mode, term-cooked-mode, term-sane-mode, signal-catch-sigint, exception-mode-pass?, exception-mode-panic?, mod-path

• (license) will print the license terms for slope, this is mostly input as a convenience in the repl.
• The various term modes are an experimental feature at the moment, but should work as described

System API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (exit [[number]]): ()
• (license): ()
• (apply [procedure] [arguments: list]): value
• (! [exception-text: string]): exception
• (exception-mode-panic): ()
• (exception-mode-pass): ()
• (exception-mode-panic?): bool
• (exception-mode-pass?): bool
• (term-size): list
• (term-restore): ()
• (term-char-mode): ()
• (term-raw-mode): ()
• (term-cooked-mode): ()
• (term-sane-mode): ()
• (signal-catch-sigint [procedure]): bool The procedure passed to signal-catch-sigint should take no arguments and will be run upon sigint being received. This will override all default behavior (if you still want the program to exit you will need to call (exit) from within the procedure). This procedure (signal-catch-sigint) should be called at the root level of your program as it will only have access to the global environment.
• (mod-path): string

OS Related

path-exists?, path-is-dir?, path-abs, path-join, path-extension, path-base, path-dir, path-glob, chmod, chdir, env, subprocess, mkdir, rm, mv, pwd

OS API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (path-exists? [filepath: string]): bool
• (path-is-dir? [filepath: string]): bool
• (path-abs [filepath: string]): string
• (path-join [string]): string
• (path-extension [filepath: string] [[replacement-extension: string]]): string
• (path-base [filepath: string]): string
• (path-dir [filepath: string]): string
• (path-glob [filepath: string]): list
• (chmod [filepath: string] [file-mode: number]): ()
• (chdir [filepath: string]): ()
• (env [[env-key: string]] [[env-value: string]]): list|string|()
• (subprocess [list] [[output-redirection: IOHandle|#f]] [[error-redirection: IOHandle|#f]]): number (Passing `#f` to output redirection allows you to do stdout while still redirecting stderr. The `#f` for stderr only exists for symetry)
• (mkdir [path: string] [permissions: number] [[make-all: bool]]): ()
• (rm [path: string] [[make-all: bool]]): ()
• (mv [from-path: string] [to-path: string]): ()
• (pwd): path: string

Net Related

Implemented:

net-conn, url-scheme, url-host, url-port, url-path, url-query, hostname

Note: For the time being, net-conn can be used to make most requests. At the moment the tls settings are not particularly secure and I am looking into ways to allow for more granular customization of both the timeout and the tls without weighing down the function or introducing more types.

Net API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (net-conn [host: string] [port: string|number] [[use-tls: bool]] [[timeout-seconds: number]]): IOHandle
• (url-scheme [url: string] [[new-scheme: string]]): string
• (url-host [url: string] [[new-host: string]]): string
• (url-port [url: string] [[new-port: string]]): string
• (url-path [url: string] [[new-path: string]]): string
• (url-query [url: string] [[new-query: string]]): string
• (hostname): string

Date/Time Related

Implemented:

timestamp, date, timestamp->date, date->timestamp, date-format, sleep, date-default-format

Date/Time API Description

Anything inside `[]` is a placeholder representing an argument. Anything inside `` is a placeholder representing an _optional_ argument. `...` indicates a variadic value.

What the procedure returns is indicated, though any procedure can return an exception in addition to the given type.

• (timestamp): timestamp: number
• (date): date: string
• (timestamp->date [timestamp: string|number] [[format: string]]): string
• (date->timestamp [date: string] [format: string]): timestamp: number
• (date-format [start-format: string] [date: string] [end-format: string]): string
• (sleep [[milliseconds: number]]): ()
• (date-default-format): string (returns a format string for the default format produced by date)

In the above procedures there are strings listed as "format". The format string is similar to the format string used in PHP, Python, etc.

Certain characters are escaped by % symbols followed by a single character. Here is the list of the conversions:

%a

12 hour segment, lowercase: pm

%A

12 hour segment, uppercase: PM

%d

Day without leading zero: 4

%D

Day with leading zero: 04

%e

Day without leading zero, but with space padding: 4

%f

Month name, short: Jan

%F

Month name, long: January

%g or %G

Hour, 24 hour format: 15

%h

Hour, 12 hour format without leading zero: 2

%H

Hour, 12 hour format with leading zero: 02

%i

Minutes without leading zero: 4

%I

Minutes with leading zero: 04

%m

Month number, without leading zero: 6

%M

Month number, with leading zero: 06

%o

Timezone offset, without minutes: -07

%O

Timezone offset, with minutes: -0700

%s

Seconds, without leading zero: 5

%S

Seconds, with leading zero: 05

%w

Short weekeday: Wed

%W

Long weekday: Wednesday

%y

year: 21

%Y

Four digit year: 2021

%Z

Time zone, as three chars: UTC, PST, etc.

%

Will yield a literal %

anything else

A percent followed by an unrecognized char will yield a ?, as will a hanging % as the last char in the string

So the string: "%F %e, %Y %h:%I%a" would be equivalend to something like: "August 8, 2021 4:03pm"

When a string is converted to a date and no time zone information is present, slope will assume the user's local time.

---

*the general write, read, and close operations in the IO section apply to net objects as well as files and string buffers

Preload

slope allows for files in a specific directory to be automatically loaded. For this to occur slope should be invoked with the -L flag. When that flag is passed slope will do its normal setup routine then preload all files in the preload folder then go on to whatever its main task is (repl, run file, run single line from command).

slope will use the variable $SLOPE_PRELOAD_DIR if it is defined and not empty. Otherwise, if $XDG_DATA_HOME is defined and not empty it will use $XDG_DATA_HOME/slope/preload/. Lastly it will use ~/.local/share/slope/preload/.

Unlike modules, arbitrary code execution will occur with preloads (not just define expressions). The rationalle is that you, the user, have put this code there and as such it is considered safe and in your control. Preloads are a really easy way to make procedures and variables outside of the standard library, but that you regularly use, available at a repl session without having to manually load them.

Modules

slope has a basic module system. When you use the load procedure you are loading a single file from a given path. Anything in that file will be run when it is loaded. Using load-mod allows you to load a module from a designated location on your system by simply passing the module's name (as represented by the directory name it is contained within).

load-mod will load modules from the first non-empty-string value it finds out of these options:

1. $SLOPE_MOD_PATH
2. $XDG_DATA_HOME/slope/modules
3. ~/.local/share/slope/modules

So, a hypethetical module named test would be found at ~/.local/share/slope/modules/test.

slp

slope has a package manager available at https://git.rawtext.club/slope-lang/slp. Once installed you will be able to search/browse packages, install, remove, and update things, generate new module skeletons, read docs, etc. The slp repository has information on how to get modules added to the package registry. This is, at present, the best way to deal with modules and is highly recommended but by no means required. The same thing can be done by finding a repository with a module and cloning it to your module path. So long as it is a valid module it will be loadable with load-mod.

How Modules are Processed

When load-mod is called the module will be located and main.slo will be parsed. All expressions are ignored except define, load-mod, and load-mod-file. Arbitrary code cannot be run (there are no side effects to load-mod or load-mod-file other than populating the global environment with the defined values from the given module).

A define expression will still be fully evaluated. As such, something like: (define twenty (+ 15 5)) is perfectly fine. But something like (display "I am in a module") will not be evaluated.

Calling load-mod from within a module allows modules to have their own dependencies. Calling load-mod-file allows modules to be made up of more files than just main.slo. More info about files can be found in the next section.

Module Files

main.slo

Taking our above example of a module named test, the bare minimum that the folder ~/.local/share/slope/modules/test should contain is a file named main.slo. When a module is loaded, this is the file that is looked for. If it is not present, loading will fail and a runtime panic will result.

Other slope source files

You may, optionally, have other code files present in the module and can load them by calling load-mod-file. This procedure should be given a path relative to the module root directory. So within our main.slo file we might call something like: (load-mod-file "./submodules/unit-tests.slo"). Something to note is that a call to load-mod-file cannot jump out of the module's root directory (doing something like ../../../some-other-code.py or /etc/something will result in a runtime panic).

module.json

Modules found in the package registry will include this file. It contains metadata related to the package including things like description, version, repository url, homepage, author, and, most importantly the dependency list (so that a modules dependencies can also be installed).

If you are making a module and do not know what should be here you can look at an existing module to get a sense of it. Or, with slp installed, you can just run slp gen and it will build out your module skeleton for you.

Building

A Makefile is available. Running make will build in the local directory. Running sudo make install will install the software (sudo may or may not be needed depending on your setup, or the functionality may be provided by a different tool such as doas). slope was written with Go 1.16, but will likely build on older versions with limited modifications.

Running

All of these assume slope is on your $PATH:

To run the repl:

slope

From the repl you can execute (license) to view the license.

To run a one liner:

slope -run "(display (+ 1 3))"

To run a file:

slope ./myfile.slo

To see command line options:

slope -h

To see version information:

slope -v

To install a module from a url to a git repo:

slope -install https://git.rawtext.club/sloum/ansi.git