# rss_item

guid: https://drewdevault.com/2018/02/28/Writing-a-wayland-compositor-part-3.html
title: Writing a Wayland Compositor, Part 3: Rendering a window
url: https://drewdevault.com/2018/02/28/Writing-a-wayland-compositor-part-3.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>This is the third in a series of articles on the subject of writing a Wayland
compositor from scratch using <a href="https://github.com/swaywm/wlroots">wlroots</a>.
Check out <a href="https://drewdevault.com/2018/02/17/Writing-a-Wayland-compositor-1.html">the first article</a>
if you haven&rsquo;t already. We left off with a Wayland server which accepts client
connections and exposes a handful of globals, but does not do anything
particularly interesting yet. Our goal today is to do something interesting -
render a window!</p>
<p>The commit that this article dissects is
<a href="https://github.com/SirCmpwn/mcwayland/commit/342b7b6">342b7b6</a>.</p>
<p>The first thing we have to do in order to render windows is establish the
<strong>compositor</strong>. The wl_compositor global is used by clients to allocate
<code>wl_surface</code>s, to which they attach <code>wl_buffer</code>s. These surfaces are just a
generic mechanism for sharing buffers of pixels with compositors, and don&rsquo;t
carry an implicit <strong>role</strong>, such as &ldquo;application window&rdquo; or &ldquo;panel&rdquo;.</p>
<p>wlroots provides an implementation of <code>wl_compositor</code>. Let&rsquo;s set aside a
reference for it:</p>
<div class="highlight"><pre class="chroma"><code class="language-diff" data-lang="diff">struct mcw_server {
        struct wl_display *wl_display;
        struct wl_event_loop *wl_event_loop;
 
        struct wlr_backend *backend;
<span class="gi">+       struct wlr_compositor *compositor;
</span><span class="gi"></span> 
        struct wl_listener new_output;
</code></pre></div><p>Then rig it up:</p>
<div class="highlight"><pre class="chroma"><code class="language-diff" data-lang="diff">        wlr_primary_selection_device_manager_create(server.wl_display);
        wlr_idle_create(server.wl_display);
 
<span class="gi">+       server.compositor = wlr_compositor_create(server.wl_display,
</span><span class="gi">+                       wlr_backend_get_renderer(server.backend));
</span><span class="gi">+
</span><span class="gi"></span>        wl_display_run(server.wl_display);
        wl_display_destroy(server.wl_display);
</code></pre></div><p>If we run mcwayface now and check out the globals with <code>weston-info</code>, we&rsquo;ll see
a wl_compositor and wl_subcompositor have appeared:</p>
<pre><code>interface: 'wl_compositor', version: 4, name: 8
interface: 'wl_subcompositor', version: 1, name: 9
</code></pre><p>You get a wl_subcompositor for free with the wlroots wl_compositor
implementation. We&rsquo;ll discuss subcompositors in a later article. Speaking of
things we&rsquo;ll discuss in another article, add this too:</p>
<div class="highlight"><pre class="chroma"><code class="language-diff" data-lang="diff">        wlr_primary_selection_device_manager_create(server.wl_display);
        wlr_idle_create(server.wl_display);
 
        server.compositor = wlr_compositor_create(server.wl_display,
                        wlr_backend_get_renderer(server.backend));
 
<span class="gi">+       wlr_xdg_shell_v6_create(server.wl_display);
</span><span class="gi">+
</span><span class="gi"></span>        wl_display_run(server.wl_display);
        wl_display_destroy(server.wl_display);
        return 0;
</code></pre></div><p>Remember that I said earlier that surfaces are just globs of pixels with no
role? xdg_shell is something that can give surfaces a role. We&rsquo;ll talk about it
more in the next article. After adding this, many clients will be able to
connect to your compositor and spawn a window. However, without adding anything
else, these windows will never be shown on-screen. You have to render them!</p>
<p>Something that distinguishes wlroots from libraries like wlc and libweston is
that wlroots does not do any rendering for you. This gives you a lot of
flexibility to render surfaces any way you like. The clients just gave you a
pile of pixels, what you do with them is up to you - maybe you&rsquo;re making a
desktop compositor, or maybe you want to draw them on an Android-style app
switcher, or perhaps your compositor arranges windows in VR - all of this is
possible with wlroots.</p>
<p>Things are about to get complicated, so let&rsquo;s start with the easy part: in
the output_frame handler, we have to get a reference to every wlr_surface we
want to render. So let&rsquo;s iterate over every surface our <code>wlr_compositor</code> is
keeping track of:</p>
<div class="highlight"><pre class="chroma"><code class="language-diff" data-lang="diff">        wlr_renderer_begin(renderer, wlr_output);

<span class="gi">+       struct wl_resource *_surface;
</span><span class="gi">+       wl_resource_for_each(_surface, &amp;server-&gt;compositor-&gt;surfaces) {
</span><span class="gi">+               struct wlr_surface *surface = wlr_surface_from_resource(_surface);
</span><span class="gi">+               if (!wlr_surface_has_buffer(surface)) {
</span><span class="gi">+                       continue;
</span><span class="gi">+               }
</span><span class="gi">+               // TODO: Render this surface
</span><span class="gi">+       }
</span><span class="gi"></span>
        wlr_output_swap_buffers(wlr_output, NULL, NULL);
</code></pre></div><p>The <code>wlr_compositor</code> struct has a member named <code>surfaces</code>, which is a list of
<code>wl_resource</code>s. A helper method is provided to produce a <code>wlr_surface</code> from its
corresponding <code>wl_resource</code>. The <code>wlr_surface_has_buffer</code> call is just to make
sure that the client has actually given us pixels to display on this surface.</p>
<p>wlroots might make you do the rendering yourself, but some tools <em>are</em> provided
to help you write compositors with simple rendering requirements:
<strong>wlr_renderer</strong>. We&rsquo;ve already touched on this a little bit, but now we&rsquo;re
going to use it for real. A little bit of OpenGL knowledge is required here. If
you&rsquo;re a complete novice with OpenGL<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>, I can recommend <a href="https://learnopengl.com/">this
tutorial</a> to help you out. Since you&rsquo;re in a hurry,
we&rsquo;ll do a quick crash course on the concepts necessary to utilize wlr_renderer.
If you get lost, just skip to the next diff and treat it as magic incantations
that make your windows appear.</p>
<p>We have a pile of pixels, and we want to put it on the screen. We can do this
with a <strong>shader</strong>. If you&rsquo;re using wlr_renderer (and mcwayface will be), shaders
are provided for you. To use our shaders, we feed them a <strong>texture</strong> (the pile
of pixels) and a <strong>matrix</strong>. If we treat every pixel coordinate on our surface
as a vector from (0, 0); top left, to (1, 1); bottom right, our goal is to
produce a matrix that we can multiply a vector by to find the final coordinates
on-screen for the pixel to be drawn to. We must project pixel coordinates from
this 0-1 system to the coordinates of our desired rectangle on screen.</p>
<p>There&rsquo;s gotcha here, however: the coordinates on-screen <em>also</em> go from 0 to 1,
instead of, for example, 0-1920 and 0-1080. To project coordinates like
&ldquo;put my 640x480 window at coordinates 100,100&rdquo; to screen coordinates, we use an
<strong>orthographic projection matrix</strong>. I know that sounds scary, but don&rsquo;t worry -
wlroots does all of the work for you. Your <code>wlr_output</code> already has a suitable
matrix called <code>transform_matrix</code>, which incorporates into it the current
resolution, scale factor, and rotation of your screen.</p>
<p>Okay, hopefully you&rsquo;re still with me. This sounds a bit complicated, but the
manifestation of all of this nonsense is fairly straightforward. wlroots
provides some tools to make it easy for you. First, we have to prepare a
<code>wlr_box</code> that represents (in output coordinates) where we want the surface to
show up.</p>
<div class="highlight"><pre class="chroma"><code class="language-diff" data-lang="diff">        struct wl_resource *_surface;
        wl_resource_for_each(_surface, &amp;server-&gt;compositor-&gt;surfaces) {
                struct wlr_surface *surface = wlr_surface_from_resource(_surface);
                if (!wlr_surface_has_buffer(surface)) {
                        continue;
                }
<span class="gd">-               // TODO: Render this surface
</span><span class="gd"></span><span class="gi">+               struct wlr_box render_box = {
</span><span class="gi">+                       .x = 20, .y = 20,
</span><span class="gi">+                       .width = surface-&gt;current-&gt;width,
</span><span class="gi">+                       .height = surface-&gt;current-&gt;height
</span><span class="gi">+               };
</span><span class="gi"></span>        }
</code></pre></div><p>Now, here&rsquo;s the great part: all of that fancy math I was just talking about can
be done with a single helper function provided by wlroots: <code>wlr_matrix_project_box</code>.</p>
<div class="highlight"><pre class="chroma"><code class="language-diff" data-lang="diff">        struct wl_resource *_surface;
        wl_resource_for_each(_surface, &amp;server-&gt;compositor-&gt;surfaces) {
                struct wlr_surface *surface = wlr_surface_from_resource(_surface);
                if (!wlr_surface_has_buffer(surface)) {
                        continue;
                }
                struct wlr_box render_box = {
                        .x = 20, .y = 20,
                        .width = surface-&gt;current-&gt;width,
                        .height = surface-&gt;current-&gt;height
                };
<span class="gi">+               float matrix[16];
</span><span class="gi">+               wlr_matrix_project_box(&amp;matrix, &amp;render_box,
</span><span class="gi">+                               surface-&gt;current-&gt;transform,
</span><span class="gi">+                               0, &amp;wlr_output-&gt;transform_matrix);
</span><span class="gi"></span>        }
</code></pre></div><p>This takes a reference to a <code>float[16]</code> to store the output matrix in, a box you
want to project, some other stuff that isn&rsquo;t important right now, and the
projection you want to use - in this case, we just use the one provided by
<code>wlr_output</code>.</p>
<p>The reason we make you understand and perform these steps is because it&rsquo;s
entirely possible that you&rsquo;ll want to do them differently in the future. This
is only the simplest case, but remember that wlroots is designed for <em>every</em>
case. Now that we&rsquo;ve obtained this matrix, we can finally render the surface:</p>
<div class="highlight"><pre class="chroma"><code class="language-diff" data-lang="diff">        struct wl_resource *_surface;
        wl_resource_for_each(_surface, &amp;server-&gt;compositor-&gt;surfaces) {
                struct wlr_surface *surface = wlr_surface_from_resource(_surface);
                if (!wlr_surface_has_buffer(surface)) {
                        continue;
                }
                struct wlr_box render_box = {
                        .x = 20, .y = 20,
                        .width = surface-&gt;current-&gt;width,
                        .height = surface-&gt;current-&gt;height
                };
                float matrix[16];
                wlr_matrix_project_box(&amp;matrix, &amp;render_box,
                                surface-&gt;current-&gt;transform,
                                0, &amp;wlr_output-&gt;transform_matrix);
<span class="gi">+               wlr_render_with_matrix(renderer, surface-&gt;texture, &amp;matrix, 1.0f);
</span><span class="gi">+               wlr_surface_send_frame_done(surface, &amp;now);
</span><span class="gi"></span>        }
</code></pre></div><p>We also throw in a <code>wlr_surface_send_frame_done</code> for good measure, which lets
the client know that we&rsquo;re done with it so they can send another frame. We&rsquo;re
done! Run mcwayface now, then the following commands:</p>
<pre><code>$ WAYLAND_DISPLAY=wayland-1 weston-simple-shm &amp;
$ WAYLAND_DISPLAY=wayland-1 gnome-terminal -- htop
</code></pre><p>To see the following beautiful image:</p>
<p><img src="https://sr.ht/y_qN.png" alt=""></p>
<p>Run any other clients you like - many of them will work!</p>
<p>We used a bit of a hack today by simply rendering all of the surfaces the
<code>wl_compositor</code> knew of. In practice, we&rsquo;re going to need to extend our
xdg_shell support (and add some other shells, too) to do this properly. We&rsquo;ll
cover this in the next article.</p>
<p>Before you go, a quick note: after this commit, I reorganized things a bit -
we&rsquo;re going to outgrow this single-file approach pretty quickly soon. Check out
That commit <a href="https://github.com/SirCmpwn/mcwayface/commit/e800facb371c42d844b858af5ced456ffd6e9d08">here</a>.</p>
<p>See you next time!</p>
<p>
    Previous &mdash;
    <a href="https://drewdevault.com/2018/02/22/Writing-a-wayland-compositor-part-2.html">
        Part 2: Rigging up the server
    </a>
</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>If you&rsquo;re not a novice, we&rsquo;ll cover more complex rendering scenarios in the future. But the short of it is that you can implement your own <code>wlr_renderer</code> that wlr_compositor can use to bind textures to the GPU and then you can do whatever you want. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/03/10/How-to-write-an-IRC-bot.html
title: How to write an IRC bot
url: https://drewdevault.com/2018/03/10/How-to-write-an-IRC-bot.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>My disdain for Slack and many other Silicon Valley chat clients is <a href="https://drewdevault.com/2015/11/01/Please-stop-using-slack.html">well
known</a>, as is my undying love for IRC.
With Slack making the news lately after their recent decision to disable the IRC
and XMPP gateways in a classic <a href="https://en.wikipedia.org/wiki/Embrace%2C_extend%2C_and_extinguish">Embrace Extend
Extinguish</a>
move, they&rsquo;ve been on my mind and I feel like writing about them more. Let&rsquo;s
compare writing a bot for Slack with writing an IRC bot.</p>
<p>First of all, let&rsquo;s summarize the process for making a Slack bot. Full details
are available in <a href="https://api.slack.com/slack-apps">their documentation</a>. The
basic steps are:</p>
<ol>
<li>Create a Slack account and &ldquo;workspace&rdquo; to host the bot (you may have already
done this step). On the free plan you can have up to 10 &ldquo;integrations&rdquo; (aka
bots). This includes all of the plug-n-play bots Slack can set up for you, so
make sure you factor that into your count. Otherwise you&rsquo;ll be heading to the
pricing page and making a case to whoever runs your budget.</li>
<li>Create a &ldquo;Slack app&rdquo; through their web portal. The app will be tied to the
company you work with now, and if you get fired you will lose the app. Make
sure you make a separate organization if this is a concern!</li>
<li>The recommended approach from here is to set up subscriptions to the &ldquo;Event
API&rdquo;, which involves standing up a web server (with working SSL) on a
consistent IP address (and don&rsquo;t forget to open up the firewall) to receive
incoming notifications from Slack. You&rsquo;ll need to handle a proprietary
challenge to verify your messages via some HTTP requests coming from Slack
which gives you info to put into HTTP headers of your outgoing requests. The
Slack docs refer to this completion of this process as &ldquo;triumphant success&rdquo;.</li>
<li>Receive some JSON in a proprietary format via your HTTP server and use some
more proprietary HTTP APIs to respond to it.</li>
</ol>
<p>Alternatively, instead of steps 3 and 4 you can use the &ldquo;Real Time Messaging&rdquo;
API, which is a websocket-based protocol that starts with an HTTP request to
Slack&rsquo;s authentication endpoint, then a follow-up HTTP request to open the
WebSocket connection. Then you set up events in a similar fashion. Refer to the
complicated table in the documentation breaking down which events work through
which API.</p>
<p>Alright, so that&rsquo;s the Slack way. How does the IRC way compare? IRC is an open
standard, so to learn about it I can just read RFC 1459, which on my system is
conveniently waiting to be read at <code>/usr/share/doc/rfc/txt/rfc1459.txt</code>. This
means I can just read it locally, offline, in the text editor of my choice,
rather than on some annoying website that calls authentication a &ldquo;triumphant
success&rdquo; and complains about JavaScript being disabled.</p>
<p>You don&rsquo;t have to read it right now, though. I can give you a summary here, like
I gave for Slack. Let&rsquo;s start by not writing a bot at all - let&rsquo;s just manually
throw some bits in the general direction of Freenode. Install netcat and run
<code>nc irc.freenode.net 6667</code>, then type this into your terminal:</p>
<pre><code>NICK joebloe
USER joebloe 0.0.0.0 joe :Joe Bloe
</code></pre><p>Hey, presto, you&rsquo;re connected to IRC! Type this in to join a channel:</p>
<pre><code>JOIN #cmpwn
</code></pre><p>Then type this to say hello:</p>
<pre><code>PRIVMSG #cmpwn :Hi SirCmpwn, I'm here from your blog!
</code></pre><p>IRC is one of the simplest protocols out there, and it&rsquo;s dead easy to write a
bot for it. If your programming language can open a TCP socket (it can), then
you can use it to write an IRC bot in 2 minutes, flat. That&rsquo;s not even to
mention that there are IRC client libraries available for every programming
language on every platform ever - I even <a href="https://github.com/SirCmpwn/ChatSharp">wrote one
myself!</a> In fact, that guy is probably
the fifth or sixth IRC library I&rsquo;ve written. They&rsquo;re so easy to write that I&rsquo;ve
lost count.</p>
<p>Slack is a walled garden. Their proprietary API is defined by them and only
implemented by them. They can and will shut off parts you depend on (like the
IRC+XMPP gateways that were just shut down). IRC is over 20 years old and
software written for it then still works now. It&rsquo;s implemented by hundreds of
clients, servers, and bots. Your CI supports it and GitHub can send commit
notifications to it. It&rsquo;s ubiquitous and free. Use it!</p>


guid: https://drewdevault.com/2018/03/17/Hack-everything-without-fear.html
title: Hack everything without fear
url: https://drewdevault.com/2018/03/17/Hack-everything-without-fear.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>We live in a golden age of open source, and it can sometimes be easy to forget
the privileges that this affords us. I&rsquo;m writing this article with vim, in a
terminal emulator called urxvt, listening to music with mpv, in a Sway desktop
session, on the Linux kernel. Supporting this are libraries like glibc or musl,
harfbuzz, and mesa. I also have the support of the AMDGPU video driver, libinput
and udev, alsa and pulseaudio.</p>
<p>All of this is open source. I can be reading the code for any of these tools
within 30 seconds, and for many of these tools I already have their code checked
out somewhere on my filesystem. It gets even better, though: these projects
don&rsquo;t just make their code available - they accept patches, too! Why wouldn&rsquo;t we
take advantage of this tremendous opportunity?</p>
<p>I often meet people who are willing to contribute to one project, but not
another. Some people will shut down when they&rsquo;re faced with a problem that
requires them to dig into territory that they&rsquo;re unfamiliar with. In Sway, for
example, it&rsquo;s often places like libinput or mesa. These tools might seem foreign
and scary - but to these people, at some point, so did Sway. In reality these
codebases are quite accessible.</p>
<p>Getting around in an unfamiliar repository can be a little intimidating, but do
it enough times and it&rsquo;ll become second nature. The same tools like gdb work
just as well on them. If you have a stacktrace for a segfault originating in
libinput, compile libinput with symbols and gdb will show you the file name and
line number of the problem. Go there and read the code! Learn how to use tools
like <code>git grep</code> to find stuff. Run <code>git blame</code> to see who wrote a confusing line
of code, and send them an email! When you find the problem, don&rsquo;t be afraid to
send a patch over instead of working around it in your own code. This is
something every programmer should be comfortable doing often.</p>
<p>Even when the leads you&rsquo;re chasing down are written in unfamiliar programming
languages or utilize even more unfamiliar libraries, don&rsquo;t despair. All
programming languages have a lot in common and huge numbers of resources are
available online. Learning just enough to understand (and fix!) a particular
problem is very possible, and something I find myself doing it all the time. You
don&rsquo;t have to be an expert in a particular programming language to invoke trial
&amp; error.</p>
<p>If you&rsquo;re similarly worried about the time investment, don&rsquo;t be. You already set
aside time to work your problem, and this is just part of that process. Yes,
you&rsquo;ll probably be spending your time differently from your expectations - more
reading code than writing code.  But how is that any less productive? The
biggest time sink in this process is all the time you spend worrying about how
much time it&rsquo;s going to take, or telling me in IRC you can&rsquo;t solve your problem
because you&rsquo;re not good enough to understand mesa or the kernel or whatever.</p>
<p>An important pastime of the effective programmer is reading and understanding
the tools you use. You should at least have a basic idea of how everything on
your system works, and in the places your knowledge is lacking you should make
it your business to study up. The more you do this, the less scary foreign code
will become, and the more productive you will be. No longer will you be stuck in
your tracks because your problem leads you away from the beaten path!</p>


guid: https://drewdevault.com/2018/03/24/Decentralize-decentralize-decentralize.html
title: Achtung! Decentralize, decentralize, decentralize!
url: https://drewdevault.com/2018/03/24/Decentralize-decentralize-decentralize.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I can hardly believe it, but the media is finally putting Facebook&rsquo;s feet to the
fire! No longer is it just the weird paranoid kids shouting at everyone to stop
giving all of their information to these companies. We need to take this bull by
the horns and drive it in a productive direction, and for that reason, it&rsquo;s time
to talk about decentralization, federation, and open source.</p>
<p><em>This article has been <a href="http://getcolorings.com/ru-decentralize">translated into
Russian</a> by <a href="http://getcolorings.com">Get
Colorings</a>.</em></p>
<p>It&rsquo;s important to remember that Facebook is not the only villain on this stage.
Did you know that Google keeps <a href="https://www.google.com/maps/timeline?pb">a map of everywhere you&rsquo;ve
been</a>? That Twitter is analyzing your
tweets just like Facebook does, and sells it to advertisers just like Facebook
does? Virtually all internet companies - Snapchat, Tinder, Uber &amp; Lift, and
even more - are spying on you and selling it to advertisers. It&rsquo;s so lucrative
and easy to do this that it&rsquo;s become an <em>industry standard practice</em>.</p>
<p>The solution to the Facebook problem is not jumping ship to another centralized
commercial platform. They will be exactly the same. The commercial model for
internet services is inherently flawed. Companies like Facebook, publicly
traded, have a legal obligation to maximize profits for their shareholders.
Private companies with investors are similarly obligated. Nowhere in the
equation does it say that they&rsquo;re obligated to do <em>anything</em> for you - the only
role you serve is to be a vehicle for exploitation.</p>
<p>You need to find services whose incentives are aligned with yours. What asks do
you have from your social media platforms? It probably starts with basic things:</p>
<ul>
<li>I want to keep up with my family and friends</li>
<li>I want my family and friends to be able to keep up with me</li>
</ul>
<p>But if you&rsquo;re smart, you might have some deeper asks:</p>
<ul>
<li>I don&rsquo;t want my personal information sold to others</li>
<li>I don&rsquo;t want to be manipulated into spending my money</li>
</ul>
<p>We might even have some asks as a <em>society</em>, too:</p>
<ul>
<li>We don&rsquo;t want to be manipulated into hating our countrymen</li>
<li>We don&rsquo;t want to have our people&rsquo;s opinions radicalized</li>
</ul>
<p>Each company I&rsquo;ve mentioned, and many more, may offer you some subset of these
promises. But <em>in every case</em>, they will have conditions:</p>
<ul>
<li><strong>We&rsquo;ll help you keep up with family and friends</strong>, or at least the subset
of them that we think makes you more profitable.</li>
<li><strong>We&rsquo;ll help your family and friends keep up with you</strong>, so long as your posts
are engaging enough to keep them looking at our ads.</li>
<li><strong>Your personal information won&rsquo;t be sold to others</strong>, unless we can get away
with it.</li>
<li><strong>You won&rsquo;t be manipulated into spending your money</strong>, unless we can
manipulate you into spending it on us.</li>
<li><strong>We won&rsquo;t manipulate you into hating your countrymen</strong>, unless it makes you
spend more time using our platform to express your hatred.</li>
<li><strong>We won&rsquo;t radicalize your opinions</strong>, at least not the ones that don&rsquo;t get
you angry enough to spend more time looking at our ads.</li>
</ul>
<p>I&rsquo;m not just being cynical here. There is no promise that a company can make to
its users that outweighs the <a href="https://legal-dictionary.thefreedictionary.com/fiduciary+duty">fiduciary
duty</a> that
<em>obligates</em> them to maximize profits by any means. The only defense of this is
legislation and consumer choice. We must pass laws that defend users and we must
choose not to engage with companies that behave like this.</p>
<p>We must do both of these things, but for now I&rsquo;m going to focus on the consumer
choice. We must throw our lot in with the alternative to these corporations -
decentralized, federated, open source platforms.</p>
<p>What do each of these terms mean?</p>
<p><em>Decentralized</em> means that the platform is, well, not <em>centralized</em>. Rather
than the control being in the hands of one company (or a single interested
party, to generalize it a bit), control is in the hands of many independent
operators.</p>
<p><em>Federated</em> refers to a means by which several service operators can
communicate with each other in standard ways. This approach prevents
platform lock-in. Email is a federated system - you can send an email from
your gmail account to your mother&rsquo;s old AOL account. Contrast this to Facebook,
where you can&rsquo;t follow your friend&rsquo;s Twitter account.</p>
<p>Finally, <em>open source</em><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> is a term used by the technology community to refer
to the free distribution of the secret sauce that makes our services tick. The
technology engineering community collectively works on these projects and freely
shares this work with everyone else.</p>
<p>The combination of all of these ideas in one piece of software is the golden
ticket to internet freedom. This is the approach to social networking taken most
famously by <a href="http://joinmastodon.org/">Mastodon</a>. Mastodon is a decentralized,
federated, and open source platform. The computing infrastructure the platform
runs on is operated by thousands of independent volunteers (decentralized),
which all communicate with each other and other software using standard
protocols (federated), and the <a href="https://github.com/tootsuite/mastodon">source
code</a> is freely available for anyone to
use and improve (open source)<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>.</p>
<p>The incentives of the operators are aligned with the incentives of the users on
Mastodon. The operator of each instance is a human being who can be easily
reached to give feedback and thanks, rather than a billionaire egomaniac who buys
an entire neighborhood so no one can bother him. Because the costs of
maintaining this social network are distributed across thousands of operators,
each one has a very low cost of operation, which is usually easily covered by
donations from the users who they support. There are no investors to please.
Just the users.</p>
<p>Mastodon fills a Twitter-like niche. There are other platforms attempting to
fill other niches - <a href="https://diasporafoundation.org/">diaspora*</a> is easily
compared to Facebook, for example.
<a href="https://github.com/Chocobozzz/PeerTube">PeerTube</a> is under development to
fulfill a YouTube-like niche, too. These platforms need our support.</p>
<p>Commercial platforms don&rsquo;t respect you. You may have grown used to skimming over
ads and content you don&rsquo;t want to see on Facebook and other platforms. It&rsquo;s an
annoyance that you&rsquo;ve internalized because, well, what else can you do? There
are no ads on Mastodon. It doesn&rsquo;t need them, and you deserve better than them.</p>
<p style="text-align: center">---</p>
<p>Remember, Facebook is not the only evil. It&rsquo;s time to discard proprietary
platforms like the manipulative trash they are.  Take the anger you&rsquo;ve felt at
Facebook these past couple of weeks and use it to embrace decentralization,
federation, and open source.</p>
<p>I know it seems a monumental task to untangle your life from these companies,
but you don&rsquo;t have to do it all at once. If this article moved you, make a todo
list right now. List each way in which you&rsquo;re tied to some platform - you use
Facebook to talk to your friends, or use gmail for your email address, your
contacts are stored on Google, you use Facebook&rsquo;s calendar for social events,
you have a Twitter account you haven&rsquo;t moved&hellip; then take on each task one at a
time. Take as much time as you need. As you research these options, if you find
the open options lacking, let the people involved know what your needs are. If
there&rsquo;s no open option at all, please <a href="mailto:sir@cmpwn.com">email me</a> about it.</p>
<p>We can do this. We can be free.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>There is some debate about the use of the term &ldquo;open source&rdquo; as opposed to another term, &ldquo;free software&rdquo;. There is a time and a place for this discussion, but it&rsquo;s not here, and our message weakens if we expose the general public to our bickering. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>There are actually several competing and compatible softwares that federate with the same social network Mastodon uses. This is very similar to how several different email providers are compatible with each other and compete to innovate together. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/04/28/KDE-Sprint-retrospective.html
title: Sway reporting in from KDE's Berlin development sprint
url: https://drewdevault.com/2018/04/28/KDE-Sprint-retrospective.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I&rsquo;m writing to you from an airplane on my way back to Philadelphia, after
spending a week in Berlin working with the KDE team. It was great to meet those
folks and work with them for a while. It&rsquo;ll take me some time to get the taste
of C++ out of my mouth, though! In all seriousness, it was a very productive
week and I feel like we have learned a lot about each other&rsquo;s projects and have
a strengthened interest in collaborating more in the future.</p>
<p>The main purpose of my trip was to find opportunities for
<a href="http://swaywm.org">sway</a> and <a href="http://kde.org">KDE</a> to work together on
improving the Linux desktop. Naturally, the main topic of discussion was
interopability of software written for each of our projects. I brought the
wlroots layer-shell protocol to the table seeking their feedback on it, as well
as reviewing how their desktop shell works today. From our discussions we found
a lot of common ground in our designs and needs, as well as room for improvement
in both of our approaches.</p>
<p>The KDE approach to their desktop shell is similar to the original sway
approach. Today, their Plasma shell uses a number of proprietary protocols which
are hacks on top of the xdg-shell protocol (for those not in the know, the
xdg-shell protocol is used to render normal desktop windows and is not designed
for use with e.g. panels) that incorporate several of the concepts they were
comfortable using on X11 in an almost 1:1 fashion. Sway never had any X11
concepts to get comfortable with, but some may not know that sway&rsquo;s panel,
wallpaper, and lock screen programs on the 0.x releases are also hacks on top of
xdg-shell that are not portable between compositors.</p>
<p>In the wlroots project (which is overseen by sway), we&rsquo;ve been developing a
new protocol designed for desktop shell components like these. In theory, it is
a more generally applicable approach to building desktop shells on Wayland than
the approach we were using before. I sat down with the KDE folks and went over
this protocol in great detail, and learned about how Plasma shell works today,
and we were happy to discover that the wlroots approach (with some minor tweaks)
should be excellently suited to Plasma shell. In addition to the layer-shell, we
reviewed several other protocols Plasma uses to build its desktop experience,
and identified more places where it makes sense for us to unify our approach.
Other subjects discussed included virtual desktops, external window management,
screen capture and pipewire, and more.</p>
<p>The upshot of this is that we believe it&rsquo;s possible to integrate the Plasma
shell with sway. Users of KDE on X11 were able to replace kwin with i3 and still
utilize the Plasma shell - a feature which was lost in the transition to
Wayland. As we continue to work together, this use-case may well be captured
again. Even KDE users who are uninterested in sway stand to benefit from this.
The hacks Plasma uses today are temporary and unmaintainable, and the
improvements to Plasma&rsquo;s codebase will make it easier to work with. Should kwin
grow stable layer-shell support, clients designed for sway will work on KDE as
well. Replacing sway&rsquo;s own similar hacks will have similar benefits for our
codebase and open the door to 3rd-party panels, lockscreens, rofi, etc.</p>
<p>I spent my time in their care working on actual code to this end. I wrote up a
C++ library that extends Qt with layer-shell support called
<a href="https://github.com/SirCmpwn/qtlayershell">qtlayershell</a>, and extended the
popular <a href="#">Latte Dock</a> KDE extension to support it. Though this work is not
complete, it works - as I write this blog post, Latte is running on my sway
session! This is good progress, but I must return my focus to wlroots soon. If
you are interested in this work, please help me complete it!</p>
<p><img src="https://drewdevault.com/img/latte-dock.png" alt=""></p>
<p>A big thanks goes to KDE for putting on this event and covering my travel costs
to attend. I hope they found it as productive as I did, and I&rsquo;m very excited
about working more with them in the future. The future of Wayland is bright!</p>


guid: https://drewdevault.com/2018/05/03/Google-embraces-extends-extinguishes.html
title: Google embraces, extends, and extinguishes
url: https://drewdevault.com/2018/05/03/Google-embraces-extends-extinguishes.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Microsoft infamously coined the euphemism &ldquo;<a href="https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish">embrace, extend,
extinguish</a>&rdquo; to
describe their strategy for disrupting markets dominated by open standards.
These days, Microsoft seems to have turned the other leaf, contributing to a
huge amount of open source and supporting open standards, and is becoming a good
citizen of the technology community. It&rsquo;s time to turn our concerns to Google.</p>
<p>Google famously &ldquo;embraced&rdquo; email on April Fool&rsquo;s day, 2004, which is of course
based on an open standard and federates with the rest of the world. If you&rsquo;ve
read the news lately, you might have seen that Google is shipping a big update
to GMail soon, which adds &ldquo;self-destructing&rdquo; emails that vanish from the
recipient&rsquo;s inbox after a time. Leaving aside that this promise is impossible to
deliver, look at the implementation - Google emails a link to a webpage with the
actual email content, and does magic in their client to make it look seamless.
Thus, they &ldquo;extend&rdquo; email. The &ldquo;extinguish&rdquo; with GMail is also well underway -
it&rsquo;s infamous for having an extremely strict spam filter for incoming emails
from people who run personal or niche mail servers.</p>
<p>Then there&rsquo;s AMP. It&rsquo;s an understatement to say Google embraced the web - but
AMP is how they enter the &ldquo;extend&rdquo; phase. AMP is a &ldquo;standard&rdquo;, but they don&rsquo;t
listen to any external feedback on it and it serves as a vehicle for keeping
users on their platform even when reading content from other websites. This is
thought to be the main intention of the service, as there are plenty of other
(and more effective) ways of rewarding lightweight pages in their search
results. The &ldquo;extinguish&rdquo; phase comes as sites that don&rsquo;t play ball get pushed
out of Google search results and into obscurity. AMP is perhaps the most blatant
of Google&rsquo;s strategies, serving only to further Google&rsquo;s agenda at the expense
of everyone else.</p>
<p>The list of grievances continues. Consider Google&rsquo;s dizzying collection of chat
applications. In its initial form, gtalk supported XMPP, an open and federated
standard for chat applications. Google dropped support for XMPP in 2014 and
continued the development of their proprietary platform up thru today&rsquo;s Hangouts
and Google Chat platforms - neither of which support any open standards. Slack
is also evidently taking cues from Google here, recently shutting down their own
IRC and XMPP bridges.</p>
<p>Google Reader&rsquo;s discontinuation fits too. RSS&rsquo;s decline was evident before
Google axed it, but killing Reader dealt a huge blow to any of RSS&rsquo;s remaining
momentum. Google said themselves they wanted to consolidate users onto the rest
of their services - none of which, I should add, support any open syndication
standards.</p>
<p>What of Google&rsquo;s role as a participant in open source? Sure, they make a lot of
software open source, but they don&rsquo;t collaborate with anyone.  They forked from
WebKit to get Apple out of the picture, and contributing to Chromium as a
non-Googler is notoriously difficult. Android is the same story - open source in
principle, but non-Googler AOSP contributors bemoan their awful approach to
external patches. It took Google over a decade to start making headway on
upstreaming their Linux patches for Android, too. Google writes papers about AI,
presumably to incentivize their academics with recognition for their work. This
is great until you notice that the crucial piece, the trained models, is always
absent.</p>
<p>For many people, the alluring convenience of Google&rsquo;s services is overwhelming.
It&rsquo;s hard to hear these things. But we must face facts: embrace, extend,
extinguish is a core part of Google&rsquo;s playbook today. It&rsquo;s important that we
work to diversify the internet and fight the monoculture they&rsquo;re fostering.</p>
<hr>
<p><strong>2018-05-04 18:12 UTC</strong>: I retract my criticism of Google&rsquo;s open source portfolio
as a whole, and acknowledge their positive impact on many projects. However, of
the projects explicitly mentioned I maintain that my criticism is valid.</p>
<p><strong>2018-05-05 11:17 UTC</strong>: Apparently the previous retraction caused some
confusion. I am <em>only</em> retracting the insinuation that Google isn&rsquo;t a good actor
in open source, namely the first sentence of paragraph 6. The rest of the
article has not been retracted.</p>


guid: https://drewdevault.com/2018/05/04/Redirecitng-stderr-of-running-process.html
title: Redirecting stderr of a running process
url: https://drewdevault.com/2018/05/04/Redirecitng-stderr-of-running-process.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>During the KDE sprint in Berlin, <a href="http://www.subdiff.de/">Roman Gilg</a> leaned
over to me and asked if I knew how to redirect the stderr of an already-running
process to a file. I Googled it and found underwhelming answers using strace and
trying to decipher the output by reading the write syscalls. Instead, I thought
a gdb based approach would work better, and after putting the pieces together
Roman insisted I wrote a blog post on the topic.</p>
<p>gdb, the GNU debugger, has two important features that make this possible:</p>
<ul>
<li>Attaching to running processes via <code>gdb -p</code></li>
<li>Executing arbitrary code in the target process space</li>
</ul>
<p>With this it&rsquo;s actually quite straightforward. The process is the following:</p>
<ol>
<li>Attach gdb to the running process</li>
<li>Run <code>compile code -- dup2(open(&quot;/tmp/log&quot;, 65), 2)</code></li>
</ol>
<p>The magic 65 here is the value of <code>O_CREAT | O_WRONLY</code> on Linux, which is easily
found with a little program like this:</p>
<div class="highlight"><pre class="chroma"><code class="language-c" data-lang="c"><span class="cp">#include</span> <span class="cpf">&lt;sys/stat.h&gt;</span><span class="cp">
</span><span class="cp">#include</span> <span class="cpf">&lt;fcntl.h&gt;</span><span class="cp">
</span><span class="cp"></span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
    <span class="n">printf</span><span class="p">(</span><span class="s">&#34;%d</span><span class="se">\n</span><span class="s">&#34;</span><span class="p">,</span> <span class="n">O_CREAT</span> <span class="o">|</span> <span class="n">O_WRONLY</span><span class="p">);</span>
    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div><p>2 is always the file descriptor assigned to stderr. What happens here is:</p>
<ol>
<li>Via <a href="https://linux.die.net/man/3/open"><code>open</code></a>, the file you want to redirect
to is created.</li>
<li>Via <a href="https://linux.die.net/man/3/dup2"><code>dup2</code></a>, stderr is overwritten with
this new file.</li>
</ol>
<p>The <code>compile code</code> gdb command will compile some arbitrary C code and run the
result in the target process, presumably by mapping some executable RAM and
loading it in, then jumping to the blob. Closing gdb (control+d) will continue
the process, and it should start writing out to the file you created.</p>
<p>There are lots of other cool (and hacky) things you can do with gdb. I once
disconnected someone from an internet radio by attaching gdb to nginx and
closing their file descriptor, for example. Thanks to Roman for giving me the
chance to write an interesting blog post on the subject!</p>


guid: https://drewdevault.com/2018/05/13/scdoc.html
title: Introducing scdoc, a man page generator
url: https://drewdevault.com/2018/05/13/scdoc.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>A man page generator is one of those tools that I&rsquo;ve said I would write for a
long time, being displeased with most of the other options. For a while I used
asciidoc, but was never fond of it. There are a few things I want to see in a
man page generator:</p>
<ol>
<li>A syntax which is easy to read and write</li>
<li>Small and with minimal dependencies</li>
<li>Designed with man pages as a first-class target</li>
</ol>
<p>All of the existing tools failed some of these criteria.
<a href="http://asciidoc.org/">asciidoc</a> hits #1, but fails #2 and #3 by being written
in XSLT+Python and targetting man pages as a second-class citizen.
<a href="http://mandoc.bsd.lv/">mdocml</a> fails #1 (it&rsquo;s not much better than writing raw
roff), and to a lesser extent also fails criteria #2<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. Another option,
<a href="https://github.com/rtomayko/ronn">ronn</a> meets criteria #1 and #3, but it&rsquo;s
written in Ruby and fails #2. All of these are fine for the niches they fill,
but not what I&rsquo;m looking for. And as for GNU info&hellip; ugh.</p>
<p><a href="https://xkcd.com/912/"><img src="https://sr.ht/nemf.png" alt=""></a></p>
<p>So, after tolerating less-than-optimal tools for too long, I eventually wrote
the man page generator I&rsquo;d been promising for years:
<a href="https://git.sr.ht/~sircmpwn/scdoc">scdoc</a>. In a nutshell, scdoc is a man page
generator that:</p>
<ul>
<li>Has an easy to read and write syntax. It&rsquo;s inspired by Markdown, but
importantly it&rsquo;s not <em>actually</em> Markdown, because Markdown is designed for
HTML and not man pages.</li>
<li>Is less than 1,000 lines of POSIX.1 C99 code with no dependencies and weighs
78 KiB statically linked against musl libc.</li>
<li>Only supports generating man pages. You can post-process the roff output if
you want it converted to something else (e.g. html).</li>
</ul>
<p>I recently migrated <a href="https://github.com/swaywm/sway/pull/1958">sway&rsquo;s manual</a> to
scdoc after adding support for generating tables to it (a feature from asciidoc
that the sway manual took advantage of). This change also removes a blocker to
localizing man pages - something that would have been needlessly difficult to do
with asciidoc. Of course, scdoc has full support for UTF-8.</p>
<p>My goal was to make a man page generator that had no more dependencies than man
itself and would be a no-brainer for projects to use to make their manual more
maintainable. Please give it a try!</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>mdocml is small and has minimal dependencies, but it has <em>runtime</em> dependencies - you need it installed to read the man pages it generates. This is Bad. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/05/27/Why-rewrite-wlc.html
title: Why did we replace wlc?
url: https://drewdevault.com/2018/05/27/Why-rewrite-wlc.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>For a little over a year, I&rsquo;ve been working with a bunch of talented C
developers to build a replacement for the <a href="https://github.com/Cloudef/wlc">wlc</a>
library. The result is <a href="https://github.com/swaywm/wlroots">wlroots</a>, and we&rsquo;re
still working on completing it and updating our software to use it. The
<a href="https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/">conventional
wisdom</a>
suggests that rewriting your code from scratch is almost never the right idea.
So why did we do it, and how is it working out? I have spoken a little about
this in the past, but we&rsquo;ll answer this question in detail today.</p>
<p>Sway will have been around for 3 years as of this August. When I started the
project, I wanted to skip some of the hard parts and get directly to
implementing i3 features. To this end, I was browsing around for libraries which
provided some of the low-level plumbing for me - stuff like DRM (Display
Resource Management) and KMS (Kernel Mode Setting), EGL and GLES wires, libinput
support, and so on. I was more interested in whatever tool could get me up to
speed and writing sway-specific code quickly. My options at this point came down
to wlc and <a href="https://github.com/michaelforney/swc">swc</a>.</p>
<p>swc&rsquo;s design is a little bit better in retrospect, but I ended up choosing wlc
for the simple reason that it had an X11 backend I could use for easier
debugging. If I had used swc, I would have been forced to work without a display
server and test everything under the DRM backend - which would have been pretty
annoying. So I chose wlc and go to work.</p>
<p>Designwise, wlc is basically a Wayland compositor with a plugin API, except you
get to write <code>main</code> yourself and the plugin API communicates entirely
in-process. wlc has its own renderer (which you cannot control) and its own
desktop with its own view abstraction (which you cannot control). You have some
events that it bubbles up for you and you can make some choices like where to
arrange windows.  However, if you just wire up some basics and run <code>wlc_init</code>,
wlc will do all of the rest of the work and immediately start accepting clients,
rendering windows, and dispatching input.</p>
<p>Over time we were able to make some small improvements to wlc, but sway 0.x
still works with these basic principles today. Though this worked well at first,
over time more and more of sway&rsquo;s bugs and limitations were reflections of
problems with wlc. A lengthy discussion on IRC and <a href="https://github.com/swaywm/sway/issues/1076">on
GitHub</a> ensued and we debated for
several weeks on how we should proceed. I was originally planning on building a
new compositor entirely in-house (similar to GNOME&rsquo;s mutter and KDE&rsquo;s kwin), and
I wanted to abstract the i3-specific functionality of sway into some kind of
plugin. Then, more &ldquo;frontends&rdquo; could be written on top of sway to add
functionality like AwesomeWM, bspwm, Xmonad, etc.</p>
<p>After some discussion among the sway team and with other Wayland compositor
projects <a href="https://github.com/way-cooler/way-cooler/issues/248">facing similar
problems</a> with wlc, I
decided that we would start developing a standalone library to replace wlc
instead, and with it allow a more diverse Wayland ecosystem to flourish.
Contrary to wlc&rsquo;s design - a Wayland compositor with some knobs - wlroots is a
set of modular tools with which you build the Wayland compositor yourself. This
design allows it to be suited to a huge variety of projects, and as a result
it&rsquo;s now being used for many different Wayland compositors, each with their own
needs and their own approach to leveraging wlroots.</p>
<p>When we started working on this, I wasn&rsquo;t sure if it was going to be successful.
Work began slowly and I knew we had a monumental task ahead of us. We spent a
lot of time and a few large refactorings getting a feel for how we wanted the
library to take shape. Different parts matured at different paces, sometimes
with changes in one area causing us to rethink design decisions that affected
the whole project. Eventually, we fell into our stride and found an approach
that we&rsquo;re very happy with today.</p>
<p>I think that the main difference with the approach that wlroots takes comes from
experience. Each of the people working on sway, wlc, way cooler, and so on were
writing Wayland compositors for the first time. I&rsquo;d say the problems that arose
as a result can also be seen throughout other projects, including Weston, KWin,
and so on. The problem is that when we all set out, we didn&rsquo;t fully understand
the opportunities afforded by Wayland&rsquo;s design, nor did we see how best to
approach tying together the rather complicated Linux desktop stack into a
cohesive project.</p>
<p>We could have continued to maintain wlc, fixed bugs, refactored parts of it, and
maybe eventually arrived at a place where sway more or less worked. But we&rsquo;d
simply be carrying on the X11 tradition we&rsquo;ve been trying to escape this whole
time. wlc was a kludge and replacing it was well worth the effort - it simply
could not have scaled to the places where wlroots is going. Today, wlroots is
the driving force behind 6 Wayland compositors and is targeting desktops,
tablets, and phones. Novel features never seen on any desktop - even beyond
Linux - are possible with this work. Now we can think about not only replacing
X11, but innovating in ways it never could have.</p>
<p>Our new approach is the way that Wayland compositors should be made. wlroots is
the realization of Wayland&rsquo;s potential. I am hopeful that our design decisions
will have a lasting positive impact on the Wayland ecosystem.</p>


guid: https://drewdevault.com/2018/05/29/Embedding-files-in-C.html
title: Embedding files in C programs with koio
url: https://drewdevault.com/2018/05/29/Embedding-files-in-C.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Quick blog post today to introduce a new tool I wrote:
<a href="https://git.sr.ht/~sircmpwn/koio">koio</a>. This is a small tool which takes a
list of files and embeds them in a C file. A library provides an fopen shim
which checks the list of embedded files before resorting to the real filesystem.</p>
<p>I made this tool for <a href="https://github.com/SirCmpwn/chopsui">chopsui</a>, where I
eventually want to be able to bundle up sui markup, stylesheets, images, and so
on in a statically linked chopsui program. Many projects have small tools which
serve a similar purpose, but it was simple enough and useful enough that I chose
to make something generic so it could be used on several projects.</p>
<p>The usage is pretty simple. I can embed <code>ko_fopen.c</code> in a C file with this
command:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">$ koio -o bundle.c ko_fopen.c://ko_fopen.c
</code></pre></div><p>I can compile and link with <code>bundle.c</code> and do something like this:</p>
<div class="highlight"><pre class="chroma"><code class="language-c" data-lang="c"><span class="cp">#include</span> <span class="cpf">&lt;koio.h&gt;</span><span class="cp">
</span><span class="cp"></span>
<span class="kt">void</span> <span class="nf">koio_load_assets</span><span class="p">(</span><span class="kt">void</span><span class="p">);</span>
<span class="kt">void</span> <span class="nf">koio_unload_assets</span><span class="p">(</span><span class="kt">void</span><span class="p">);</span>

<span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
    <span class="n">koio_load_assets</span><span class="p">();</span>
    <span class="n">FILE</span> <span class="o">*</span><span class="n">src</span> <span class="o">=</span> <span class="n">ko_fopen</span><span class="p">(</span><span class="s">&#34;//ko_fopen.c&#34;</span><span class="p">,</span> <span class="s">&#34;r&#34;</span><span class="p">);</span>
    <span class="kt">int</span> <span class="n">c</span><span class="p">;</span>
    <span class="k">while</span> <span class="p">((</span><span class="n">c</span> <span class="o">=</span> <span class="n">fgetc</span><span class="p">(</span><span class="n">src</span><span class="p">))</span> <span class="o">!=</span> <span class="n">EOF</span><span class="p">)</span> <span class="p">{</span>
        <span class="n">putchar</span><span class="p">(</span><span class="n">c</span><span class="p">);</span>
    <span class="p">}</span>
    <span class="n">fclose</span><span class="p">(</span><span class="n">src</span><span class="p">);</span>
    <span class="n">koio_unload_assets</span><span class="p">();</span>
    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div><p>The generated <code>bundle.c</code> looks like this:</p>
<div class="highlight"><pre class="chroma"><code class="language-c" data-lang="c"><span class="cp">#include</span> <span class="cpf">&lt;koio.h&gt;</span><span class="cp">
</span><span class="cp"></span>
<span class="k">static</span> <span class="k">struct</span> <span class="p">{</span>
	<span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="n">path</span><span class="p">;</span>
	<span class="n">size_t</span> <span class="n">len</span><span class="p">;</span>
	<span class="kt">char</span> <span class="o">*</span><span class="n">data</span><span class="p">;</span>
<span class="p">}</span> <span class="n">files</span><span class="p">[]</span> <span class="o">=</span> <span class="p">{</span>
	<span class="p">{</span>
		<span class="p">.</span><span class="n">path</span> <span class="o">=</span> <span class="s">&#34;//ko_fopen.c&#34;</span><span class="p">,</span>
		<span class="p">.</span><span class="n">len</span> <span class="o">=</span> <span class="mi">408</span><span class="p">,</span>
		<span class="p">.</span><span class="n">data</span> <span class="o">=</span>
<span class="s">&#34;#define _POSIX_C_SOURCE 200809L</span><span class="se">\n</span><span class="s">#include &lt;errno.h&gt;</span><span class="se">\n</span><span class="s">#include &lt;stdlib.h&gt;</span><span class="se">\n</span><span class="s">#inc&#34;</span>
<span class="s">&#34;lude &lt;stdio.h&gt;</span><span class="se">\n</span><span class="s">#include </span><span class="se">\&#34;</span><span class="s">koio_private.h</span><span class="se">\&#34;\n\n</span><span class="s">FILE *ko_fopen(const char *path&#34;</span>
<span class="s">&#34;, const char *mode) {</span><span class="se">\n\t</span><span class="s">struct file_entry *entry = hashtable_get(&amp;koio_vfs, p&#34;</span>
<span class="s">&#34;ath);</span><span class="se">\n\t</span><span class="s">if (entry) {</span><span class="se">\n\t\t</span><span class="s">if (mode[0] != &#39;r&#39; || mode[1] != &#39;</span><span class="se">\\</span><span class="s">0&#39;) {</span><span class="se">\n\t\t\t</span><span class="s">er&#34;</span>
<span class="s">&#34;rno = ENOTSUP;</span><span class="se">\n\t\t\t</span><span class="s">return NULL;</span><span class="se">\n\t\t</span><span class="s">}</span><span class="se">\n\t\t</span><span class="s">return fmemopen(entry-&gt;data, en&#34;</span>
<span class="s">&#34;try-&gt;len, </span><span class="se">\&#34;</span><span class="s">r</span><span class="se">\&#34;</span><span class="s">);</span><span class="se">\n\t</span><span class="s">}</span><span class="se">\n\t</span><span class="s">return fopen(path, mode);</span><span class="se">\n</span><span class="s">}</span><span class="se">\n</span><span class="s">&#34;</span><span class="p">,</span>
	<span class="p">},</span>
<span class="p">};</span>

<span class="kt">void</span> <span class="nf">koio_load_assets</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span> <span class="p">{</span>
	<span class="n">ko_add_file</span><span class="p">(</span><span class="n">files</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">path</span><span class="p">,</span> <span class="n">files</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">data</span><span class="p">,</span> <span class="n">files</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">len</span><span class="p">);</span>
<span class="p">}</span>

<span class="kt">void</span> <span class="nf">koio_unload_assets</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span> <span class="p">{</span>
	<span class="n">ko_del_file</span><span class="p">(</span><span class="n">files</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">path</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div><p>A very simple tool, but one that I hope people will find useful. It&rsquo;s very
lightweight:</p>
<ul>
<li>312 lines of C</li>
<li>/bin/koio is ~40 KiB statically linked to musl</li>
<li>libkoio.a is ~18 KiB</li>
<li>Only mandatory dependencies are POSIX 2008 and a C99 compiler</li>
<li>Only optional dependency is <a href="https://git.sr.ht/~sircmpwn/scdoc">scdoc</a> for the
manual, which is similarly lightweight</li>
</ul>
<p>Enjoy!</p>


guid: https://drewdevault.com/2018/06/01/How-I-maintain-FOSS-projects.html
title: How I maintain FOSS projects
url: https://drewdevault.com/2018/06/01/How-I-maintain-FOSS-projects.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Today&rsquo;s is another blog post which has been on my to-write list for a while. I
have hesitated a bit to write about this, because I&rsquo;m certain that my approach
isn&rsquo;t perfect. I think it&rsquo;s pretty good, though, and people who work with me in
FOSS agreed after a quick survey. So! Let&rsquo;s at least put it out there and
discuss it.</p>
<p>There are a few central principles I use to guide my maintainership work:</p>
<ol>
<li>Everyone is a volunteer and should be treated as such.</li>
<li>One patch is worth a thousand bug reports.</li>
<li>Empower people to do what they enjoy and are good at.</li>
</ol>
<p>The first point is very important. My open source projects are not the work of a
profitable organization which publishes open source software as a means of
giving back. Each of these projects is built and maintained entirely by
volunteers. Acknowledging this is important for keeping people interested in
working on the project - you can never expect someone to volunteer for work they
aren&rsquo;t enjoying<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. I am always grateful for any level of involvement a person
wants to have in the project.</p>
<p>Because everyone is a volunteer, I encourage people to work on their own
agendas, on their own schedule and at their own pace. None of our projects are
in a hurry, so if someone is starting to get burnt out, they should have no
reservations about taking a break for as long as they wish. I&rsquo;d rather have
something done slowly, correctly, and by a contributor who is enjoying their
work than quickly and by a contributor who is burnt out and stressed. No one
should ever be stressed out because of their involvement in the project. Some of
it is unavoidable - especially where politics is involved - but I don&rsquo;t hold
grudges against anyone who steps away and I try to shoulder the brunt of the
bullshit myself.</p>
<p>The second principle is closely related to the first. If a bug does not affect
someone who works on the project and the problem doesn&rsquo;t interest anyone who
works on the project, it&rsquo;s probably not going to get fixed. I would much rather
help someone familiarize themselves with the codebase and tooling necessary for
them to solve their own problems and send a patch, even if it takes ten times
longer than fixing the bug myself. I have never found a user who, even if they
aren&rsquo;t comfortable with programming or the specific technologies in use, has
been unable to solve a problem which they were willing to invest time into and
ask questions about.</p>
<p>This principle often leads to conflict with users whose bugs don&rsquo;t get fixed,
but I stick to it. I would rather lose every user who is unwilling to attempt a
patch than invest the resources of my contributors into work they&rsquo;re
uninterested in. In the long term, the health of the project is far better if I
always have developers engaged in and enjoying their work on it than if I lose
users who are upset by my approach.</p>
<p>These first two principles don&rsquo;t affect my day-to-day open source work so much
as they set the tone for it. The third principle, however, constitutes most of
my job as a maintainer, and it&rsquo;s with it that I add the most value. My main role
is to empower people who contribute to do work they enjoy, which benefits the
project, and which keeps them interested in coming back to do more.</p>
<p>Finding things people enjoy working on is the main task in this role. Once
people have made a few contributions, I can get an idea of how they like to work
and what they&rsquo;re good at, and help them find things to do which play to their
strengths. Supporting a contributors potential is important as well, and if
someone expresses interest in certain kinds of work or I think they show promise
in an area, it&rsquo;s my responsibility to help them find work to nurture these
skills and connect them with good mentors to help.</p>
<p>This starts to play in another major responsibility I have as a maintainer,
which is facilitating effective communication throughout the project. As people
grow in the project they generally become effective at managing communication
themselves, but new contributors appear all the time. A major responsibility as
a maintainer is connecting new contributors to domain experts in a problem, or
to users who can reproduce problems or are willing to test their patches.</p>
<p>I&rsquo;m also responsible for keeping up with each contributor&rsquo;s growth in the
project. For those who are good at and enjoy having responsibility in the
project, I try to help them find it. As contributors gain a better understanding
of the code, they&rsquo;re trusted to handle large features with less handholding and
perform more complex work<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>. Often contributors are given opportunities to
become better code reviewers, and usually get merge rights once they&rsquo;re good at
it. Things like commit access are a never a function of rank or status, but of
enabling people to do the things that they&rsquo;re good at.</p>
<p>It&rsquo;s also useful to remember that your projects are not the only game in town. I
frequently encourage people who contribute to contribute to other projects as
well, and I personally try to find ways to contribute back to their own projects
(though not as much as I&rsquo;d often like to). I offer support as a sysadmin to many
projects started by contributors to my projects and I send patches whenever I
can. This pays directly back to the project in the form of contributors with
deeper and more diverse experience. It&rsquo;s also fun to take a break from working
on the same stuff all the time!</p>
<p>There&rsquo;s also some work that someone&rsquo;s just gotta do, and that someone is usually
me. I have to be a sysadmin for the websites, build infrastructure, and so on.
If there are finances, I have to manage them. I provide some kind of vision for
the project and decide what work is in scope. There&rsquo;s also some boring stuff
like preparing changelogs and release notes and shipping new versions, or
liaising with distros on packages. I also end up being responsible for any
marketing.</p>
<hr>
<p>Getting and supporting contributors is the single most important thing you can
do for your project as a maintainer. I often get asked how I&rsquo;m as productive as
I seem to be. While I can&rsquo;t deny that I can write a lot of code, it&rsquo;s peanuts
compared to the impact made by other contributors. I get a lot of credit for
sway, but in reality I&rsquo;ve only written 1-3 sway commits per week in the past few
months. For this reason, the best approach focuses on the contributors, to whom
I owe a great debt of gratitude.</p>
<p>I&rsquo;m still learning, too! I speak to contributors about my approach from time to
time and ask for feedback, and I definitely make mistakes. I hope that I&rsquo;ll
receive more feedback soon after some of them read this blog post, too. My
approach will continue to grow over time (hopefully for the better) and I hope
our work will enjoy success as a result.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Some people do work they don&rsquo;t enjoy out of gratitude to the project, but this is not sustainable and I discourage it. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Though I always encourage people to work on the things they&rsquo;re interested in, I sometimes have to <em>discourage</em> people from biting off more than they can chew. Then I help them gradually ramp up their skills and trust among the team until they can take on those tasks. Usually this goes pretty quick, though, and a couple of bugs caused by inexperience is a small price to pay for the <em>gain</em> in experience the contributor gets by taking on hard or important tasks. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/06/05/Should-you-move-to-sr.ht.html
title: Should you move from GitHub to sr.ht
url: https://drewdevault.com/2018/06/05/Should-you-move-to-sr.ht.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I&rsquo;m not terribly concerned about Microsoft&rsquo;s acquisition of GitHub, but I
don&rsquo;t fault those who are worried. I&rsquo;ve been working on my alternative platform,
<a href="https://sr.ht">sr.ht</a>, for quite a while. I&rsquo;m not about to leave GitHub because
of Microsoft alone. I do have some political disagreements with GitHub and
Microsoft, but those are also not the main reason that I&rsquo;m building sr.ht. I
simply think I can do it better. If my approach aligns with your needs, then
sr.ht may be the platform for you.</p>
<p>There are several GitHub alternatives, but for the most part they&rsquo;re basically
GitHub rip-offs. Unlike GitLab, Gogs/Gitea, BitBucket; I don&rsquo;t see the GitHub UX
as the pinnacle of project hosting - there are many design choices (notably pull
requests) which I think have lots of room for improvement. sr.ht instead
embraces git more closely, for example building <em>on top</em> of email rather than
<em>instead of</em> email.</p>
<p>GitHub optimizes for the end-user and the drive-by contributor. sr.ht optimizes
for the maintainers and core contributors instead. We have patch queues and
ticket queues which you can set up automated filters in or manually curate, and
are reusable for projects on external platforms. You have tools which allow
you to customize the views you see separately from the views visitors see, like
bugzilla-style custom ticket searches. Our CI service gives you KVM
virtualization and knobs you can tweak to run sophisticated automation for your
project. Finally, all of it is <a href="https://git.sr.ht/~sircmpwn/?search=sr.ht">open
source</a>.</p>
<p>The business model is also something I think I can do better. GitHub and GitLab
are both VC-funded and trapped into appeasing their shareholders (or now, in
GitHub&rsquo;s case, the needs of Microsoft as a whole). I think this leads to
incentives which don&rsquo;t align with the users, as it&rsquo;s often more important to
support the bottom line than to build what the users want or need. Rather than
trying to raise as much money as possible, the sr.ht aims to be more a
grassroots platform. I&rsquo;m still working on the money details, but each user will
be expected to pay a subscription fee and growth will be artificially slowed if
necessary to make sure the infrastructure can keep up. In my opinion, venture
capital does not lead to healthy businesses or a healthy economy on the whole,
and I think the users suffer for it. My approach is different.</p>
<p>As for my own projects and the plan for moving them, I don&rsquo;t intend to move
anything until it won&rsquo;t be disruptive to the project. I&rsquo;ve been collecting
feedback from co-maintainers and core contributors to each of the projects I
expect to move and using this feedback to drive sr.ht priorities. They will
eventually move, but only when it&rsquo;s ready.</p>
<p>I intend to open sr.ht to the public soon, once I have a billing system in place
and break ground on mailing lists (among some smaller improvements). If anyone
is interested in checking it out prior to the public release, shoot me an email
at <a href="mailto:sir@cmpwn.com">sir@cmpwn.com</a>.</p>


guid: https://drewdevault.com/2018/06/27/My-lets-encrypt-setup.html
title: A quick review of my Let's Encrypt setup
url: https://drewdevault.com/2018/06/27/My-lets-encrypt-setup.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Let&rsquo;s Encrypt makes TLS much easier for pretty much everyone, but can still
be annoying to use. It took me a while to smooth over the cracks in my Let&rsquo;s
Encrypt configuration across my (large) fleet of different TLS-enabled services.
I wanted to take a quick moment to share setup with you.</p>
<p>2020-01-02 update: acme-client is unmaintained and caught the BSD disease
anyway. I use <a href="https://github.com/ndilieto/uacme">uacme</a> and my current
procedure is documented on my <a href="https://drewdevault.com/new-server.html">new server checklist</a>. It might
not be exactly applicable to your circumstances, YMMV.</p>
<p>The main components are:</p>
<ul>
<li><a href="https://kristaps.bsd.lv/acme-client/">acme-client</a></li>
<li>nginx</li>
<li>cron</li>
</ul>
<p>nginx and cron need no introduction, but acme-client deserves a closer look. The
acme client blessed by Let&rsquo;s Encrypt is <a href="https://certbot.eff.org/">certbot</a>, but
BOY is it complicated. It&rsquo;s a big ol' pile of Python and I&rsquo;ve found it fragile,
complicated, and annoying. The goal of maintaining your nginx and apache configs
for you is well intentioned but ultimately useless for advanced users. The
complexity of certbot is through the roof, and complicated software breaks.</p>
<p>I bounced between alternatives for a while but when I found acme-client, it
totally clicked. This one is written in C with minimal dependencies (LibreSSL
and libcurl, no brainers IMO). I bring a statically linked acme-client binary
with me to new servers and setup time approaches zero as a result.</p>
<p>I use nginx to answer challenges (and for some services, to use the final
certificates for HTTPS - did you know you can use Let&rsquo;s Encrypt for more
protocols than just HTTPS?). I quickly <code>mkdir -p /var/www/acme/.well-known/acme-challenge</code>, make sure nginx can read it, and add
the following rules to nginx to handle challenges:</p>
<div class="highlight"><pre class="chroma"><code class="language-nginx" data-lang="nginx"><span class="k">server</span> <span class="p">{</span>
    <span class="kn">listen</span> <span class="mi">80</span><span class="p">;</span>
    <span class="kn">listen</span> <span class="s">[::]:80</span><span class="p">;</span>
    <span class="kn">server_name</span> <span class="s">example.org</span><span class="p">;</span>

    <span class="kn">location</span> <span class="s">^~</span> <span class="s">/.well-known/acme-challenge</span> <span class="p">{</span>
        <span class="kn">alias</span> <span class="s">/var/www/acme</span><span class="p">;</span>
    <span class="p">}</span>
<span class="p">}</span>
</code></pre></div><p>If I&rsquo;m not using the certificates for HTTPS, this is all I need. But assuming I
have some kind of website going, the full configuration usually looks more like
this:</p>
<div class="highlight"><pre class="chroma"><code class="language-nginx" data-lang="nginx"><span class="k">server</span> <span class="p">{</span>
    <span class="kn">listen</span> <span class="mi">80</span><span class="p">;</span>
    <span class="kn">listen</span> <span class="s">[::]:80</span><span class="p">;</span>
    <span class="kn">server_name</span> <span class="s">example.org</span><span class="p">;</span>

    <span class="kn">location</span> <span class="s">/</span> <span class="p">{</span>
        <span class="kn">return</span> <span class="mi">302</span> <span class="s">https://</span><span class="nv">$server_name$request_uri</span><span class="p">;</span>
    <span class="p">}</span>

    <span class="kn">location</span> <span class="s">^~</span> <span class="s">/.well-known/acme-challenge</span> <span class="p">{</span>
        <span class="kn">alias</span> <span class="s">/var/www/acme</span><span class="p">;</span>
    <span class="p">}</span>
<span class="p">}</span>

<span class="k">server</span> <span class="p">{</span>
    <span class="kn">listen</span> <span class="mi">443</span> <span class="s">ssl</span><span class="p">;</span>
    <span class="kn">listen</span> <span class="s">[::]:443</span> <span class="s">ssl</span><span class="p">;</span>
    <span class="kn">server_name</span> <span class="s">example.org</span><span class="p">;</span>

    <span class="kn">ssl_certificate</span> <span class="s">/etc/ssl/acme/</span><span class="nv">$server_name/fullchain.pem</span><span class="p">;</span>
    <span class="kn">ssl_certificate_key</span> <span class="s">/etc/ssl/acme/</span><span class="nv">$server_name/privkey.pem</span><span class="p">;</span>

    <span class="kn">location</span> <span class="s">^~</span> <span class="s">/.well-known/acme-challenge</span> <span class="p">{</span>
        <span class="kn">alias</span> <span class="s">/var/www/acme</span><span class="p">;</span>
    <span class="p">}</span>

    <span class="c1"># ...application specific rules...
</span><span class="c1"></span><span class="p">}</span>
</code></pre></div><p>This covers the nginx side of things. To actually do certificate negotiation, I
have a simple script I carry around:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh"><span class="nb">exec</span> &gt;&gt;/var/log/acme 2&gt;<span class="p">&amp;</span><span class="m">1</span>
date

acme<span class="o">()</span> <span class="o">{</span>
    <span class="nv">site</span><span class="o">=</span><span class="nv">$1</span>
    <span class="nb">shift</span>
    acme-client -vNn <span class="se">\
</span><span class="se"></span>        -c /etc/ssl/acme/<span class="nv">$site</span>/ <span class="se">\
</span><span class="se"></span>        -k /etc/ssl/acme/<span class="nv">$site</span>/privkey.pem <span class="se">\
</span><span class="se"></span>        <span class="nv">$site</span> <span class="nv">$*</span>
<span class="o">}</span>

acme example.org subd1.example.org subd2.example.org

nginx -s reload
</code></pre></div><p>The first two lines set up a log file in <code>/var/log/acme</code> I can use to debug any
issues that arise. Then I have a little helper function that wires up
acme-client the way I like it, and I can call it for each domain I need certs
for on this server. The last line changes if I&rsquo;m doing something other than
HTTPS with the certs (for example, <code>postfix reload</code>).</p>
<p>One gotcha is that acme-client will bail out if the directories don&rsquo;t exist when
you run it, so a quick <code>mkdir -p /etc/ssl/acme/example.org</code> when adding new
sites is necessary</p>
<p>The final step is a simple cron entry that runs the script daily:</p>
<pre><code class="language-cron" data-lang="cron">0 0 * * * /usr/local/bin/acme-update-certs
</code></pre><p>It&rsquo;s that easy. It took me a while to get a Let&rsquo;s Encrypt setup that was simple
and satisfactory, but I believe I&rsquo;ve settled on this one. I hope you find it
useful!</p>


guid: https://drewdevault.com/2018/07/02/Email-driven-git.html
title: The advantages of an email-driven git workflow
url: https://drewdevault.com/2018/07/02/Email-driven-git.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p><a href="https://raw.githubusercontent.com/git/git/master/Documentation/RelNotes/2.18.0.txt">git 2.18.0</a> has been released, and with it my first contribution to
git has shipped! My patch was for a git feature which remains disappointingly
obscure: <a href="https://git-scm.com/docs/git-send-email">git send-email</a>. I want to
introduce my readers to this feature and speak to the benefits of using an
email-driven git workflow - the workflow git was originally designed for.</p>
<p>Email isn&rsquo;t as sexy as GitHub (and its imitators), but it has several
advantages over the latter. Email is standardized, federated, well-understood,
and venerable. A very large body of email-related software exists and is equally
reliable and well-understood. You can interact with email using only open source
software and customize your workflow at every level of the stack - filtering,
organizing, forwarding, replying, and so on; in any manner you choose.</p>
<p>Git has several built-in tools for leveraging email. The first one of note is
<a href="https://git-scm.com/docs/git-format-patch">format-patch</a>. This can take a git
commit (or series of commits) and format them as plaintext emails with embedded
diffs. Here&rsquo;s a small example of its output:</p>
<pre><code class="language-mail" data-lang="mail">From 8f5045c871c3060ff5f5f99ce1ada09f4b4cd105 Mon Sep 17 00:00:00 2001
From: Drew DeVault &lt;sir@cmpwn.com&gt;
Date: Wed, 2 May 2018 08:59:27 -0400
Subject: [PATCH] Silently ignore touch_{motion,up} for unknown ids

---
 types/wlr_seat.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/types/wlr_seat.c b/types/wlr_seat.c
index f77a492d..975746db 100644
--- a/types/wlr_seat.c
+++ b/types/wlr_seat.c
@@ -1113,7 +1113,6 @@ void wlr_seat_touch_notify_up(struct wlr_seat *seat, uint32_t time,
 	struct wlr_seat_touch_grab *grab = seat-&gt;touch_state.grab;
 	struct wlr_touch_point *point = wlr_seat_touch_get_point(seat, touch_id);
 	if (!point) {
-		wlr_log(L_ERROR, &quot;got touch up for unknown touch point&quot;);
 		return;
 	}
 
@@ -1128,7 +1127,6 @@ void wlr_seat_touch_notify_motion(struct wlr_seat *seat, uint32_t time,
 	struct wlr_seat_touch_grab *grab = seat-&gt;touch_state.grab;
 	struct wlr_touch_point *point = wlr_seat_touch_get_point(seat, touch_id);
 	if (!point) {
-		wlr_log(L_ERROR, &quot;got touch motion for unknown touch point&quot;);
 		return;
 	}
 
-- 
2.18.0

</code></pre><p>git format-patch is at the bottom of git&rsquo;s stack of outgoing email features. You
can send the emails it generates manually, but usually you&rsquo;ll use git send-email
instead. It logs into the SMTP server of your choice and sends the email for
you, after running git format-patch for you and giving you an opportunity to
make any edits you like. Given that most popular email clients these days are
awful and can&rsquo;t handle basic tasks like &ldquo;sending email&rdquo; properly, I strongly
recommend this tool over attempting to send format-patch&rsquo;s output yourself.</p>
<p><img style="max-width: 75%" src="https://sr.ht/wmKv.jpg" /></p>
<p style="text-align: center; max-width: 80%; margin: 1rem auto">
    <em>
        I put a notch in my keyboard for each person who ignores my advice,
        struggles through sending emails manually, and eventually comes around
        to letting git send-email do it for them.
    </em>
</p>
<p>I recommend a few settings to apply to git send-email to make your workflow a
bit easier. One is <code>git config --global sendemail.verify off</code>, which turns off
a sometimes-annoying and always-confusing validation step which checks for
features only supported by newer SMTP servers - newer, in this case, meaning
more recent than November of 1995. I started a thread on the git mailing list
this week to discuss changing this option to off by default.</p>
<p>You can also set the default recipient for a given repository by using a local
git config: <code>git config sendemail.to admin@example.org</code>. This lets you skip a
step if you send your patches to a consistent destination for that project, like
a mailing list. I also recommend <code>git config --global sendemail.annotate yes</code>,
which will always open the emails in your editor to allow you to make changes
(you can get this with <code>--annotate</code> if you don&rsquo;t want it every time).</p>
<p>The main edit you&rsquo;ll want to make when annotating is to provide what some call
&ldquo;timely commentary&rdquo; on your patch. Immediately following the <code>---</code> after your
commit message, you can add a summary of your changes which can be seen by the
recipient, but doesn&rsquo;t appear in the final commit log. This is a useful place to
talk about anything useful regarding the testing, review, or integration of your
changes. You may also want to edit the <code>[PATCH]</code> text in the subject line to
something like <code>[PATCH v2]</code> - this can also be done with the <code>-v</code> flag as well.
I also like to add additional To&rsquo;s, Cc&rsquo;s, etc at this time.</p>
<p>Git also provides tools for the recipient of your messages. One such tool is
<a href="https://git-scm.com/docs/git-am">git am</a>, which accepts an email prepared with
format-patch and integrates it into their repository. Several flags are provided
to assist with common integration activities, like signing off on the commit or
attempting a 3-way merge. The difficult part can be getting the email to git am
in the first place. If you simply use the GMail web UI, this can be difficult. I
use <a href="http://www.mutt.org/">mutt</a>, a TUI email client, to manage incoming
patches. This is useful for being able to compose replies with vim rather than
fighting some other mail client to write emails the way I want, but more
importantly it has the <code>|</code> key, which prompts you for a command to pipe the
email into. Other tools like <a href="http://www.offlineimap.org/">OfflineIMAP</a> are also
useful here.</p>
<p>On the subject of composing replies, reviewing patches is quite easy with the
email approach as well. Many bad, yet sadly popular email clients have
popularized the idea that the sender&rsquo;s message is immutable, encouraging you to
<a href="https://en.wikipedia.org/wiki/Posting_style#Top-posting">top post</a> and leave an endlessly growing chain of replies
underneath your message. A secret these email clients have kept from you is that
you are, in fact, permitted by the mail RFCs to edit the sender&rsquo;s message as you
please when replying - a style called <a href="https://en.wikipedia.org/wiki/Posting_style#Bottom-posting">bottom posting</a>. I
strongly encourage you to get comfortable doing this in general, but it&rsquo;s
essential when reviewing patches received over email.</p>
<p>In this manner, you can dissect the patch and respond to specific parts of it
requesting changes or clarifications. It&rsquo;s just email - you can reply, forward
the message, Cc interested parties, start several chains of discussion, and so
on. I recently sent the following feedback on a patch I received:</p>
<pre><code class="language-mail" data-lang="mail">Date: Mon, 11 Jun 2018 14:19:22 -0400
From: Drew DeVault &lt;sir@cmpwn.com&gt;
To: Gregory Mullen &lt;omitted&gt;
Subject: Re: [PATCH 2/3 todo] Filter private events from events feed

On 2018-06-11  9:14 AM, Gregory Mullen wrote:
&gt; diff --git a/todosrht/alembic/versions/cb9732f3364c_clear_defaults_from_tickets_to_support_.py b/todosrht/alembic/versions/cb9732f3364c_clear_defaults_from_tickets_to_support_.py
&gt; -%&lt;-
&gt; +class FlagType(types.TypeDecorator):

I think you can safely import the srht FlagType here without implicating
the entire sr.ht database support code

&gt; diff --git a/todosrht/blueprints/html.py b/todosrht/blueprints/html.py
&gt; -%&lt;-
&gt; +def collect_events(target, count):
&gt; +    events = []
&gt; +    for e in EventNotification.query.filter(EventNotification.user_id == target.id).order_by(EventNotification.created.desc()):

80 cols

I suspect this 'collect_events' function can be done entirely in SQL
without having to process permissions in Python and do several SQL
round-trips

&gt;  @html.route(&quot;/~&lt;username&gt;&quot;)
&gt;  def user_GET(username):
&gt; -    print(username)

Whoops! Nice catch.

&gt;      user = User.query.filter(User.username == username.lower()).first()
&gt;      if not user:
&gt;          abort(404)
&gt;      trackers, _ = get_tracker(username, None)
&gt;      # TODO: only show public events (or events the current user can see)

Can remove the comment
</code></pre><p>Obviously this isn&rsquo;t the whole patch we&rsquo;re seeing - I&rsquo;ve edited it down to just
the parts I want to talk about. I also chose to leave the file names in to aid
in navigating my feedback, with casual <code>-%&lt;-</code> symbols indicating where I had
trimmed out parts of the patch. This approach is common and effective.</p>
<p>The main disadvantage of email driven development is that some people are more
comfortable working with email in clients which are not well-suited to this kind
of work. Popular email clients have caused terrible ideas like HTML email to
proliferate, not only enabling spam, privacy leaks, and security
vulnerabilities, but also making it more difficult for people to write emails
that can be understood by git or tolerated by advanced email users.</p>
<p>I don&rsquo;t think that the solution to these problems is to leave these powerful
tools hanging in the wind and move to less powerful models like GitHub&rsquo;s pull
requests. This is why on my own platform, <a href="https://sr.ht">sr.ht</a>, I chose to
embrace git&rsquo;s email-driven approach, and extend it with new tools that make it
easier to participate without directly using email. For those like me, I still
want the email to be there so you can dig my heels in and do it old-school, but
I appreciate that it&rsquo;s not for everyone.</p>
<p>I started working on the sr.ht mailing list service a couple of weeks ago, which
is where these goals will be realized with new email-driven code review tools.
My friend <a href="https://emersion.fr">Simon</a> has been helping out with a Python module
named <a href="https://git.sr.ht/~emersion/python-emailthreads/">emailthreads</a> which can
be used to parse email discussions - with a surprising degree of accuracy,
considering the flexibility of email. Once I get these tools into a usable
state, we&rsquo;ll likely see sr.ht registrations finally opened to the general public
(interested in trying it earlier? <a href="mailto:sir@cmpwn.com">Email me</a>). Of course,
it&rsquo;s all <a href="https://git.sr.ht/~sircmpwn/?search=sr.ht">open source</a>, so you can
follow along and try it on your own infrastructure if you like.</p>
<p>Using email for git scales extremely well. The canonical project, of course, is
the Linux kernel. A change is made to the Linux kernel an average of 7 times per
hour, constantly. It is maintained by dozens of veritable clans of software
engineers hacking on dozens of modules, and email allows these changes to
efficiently flow code throughout the system. Without email, Linux&rsquo;s maintenance
model would be impossible. It&rsquo;s worth noting that git was designed for
maintaining Linux, of course.</p>
<p>With the right setup, it&rsquo;s well suited to small projects as well. Sending a
patch along for review is a single git command. It lands directly in the
maintainer&rsquo;s inbox and can be integrated with a handful of keystrokes. All of
this works without any centralization or proprietary software involved. We
should embrace this!</p>
<hr>
<p>Related articles sent in by readers:</p>
<p><a href="https://begriffs.com/posts/2018-06-05-mailing-list-vs-github.html">Mailing lists vs Github</a>
by Joe Nelson</p>
<p><a href="https://web.archive.org/web/20180522180815/https://dpc.pw/blog/2017/08/youre-using-git-wrong/">You&rsquo;re using git wrong</a> by
Dawid Ciężarkiewicz</p>


guid: https://drewdevault.com/2018/07/09/Simple-correct-fast.html
title: Simple, correct, fast: in that order
url: https://drewdevault.com/2018/07/09/Simple-correct-fast.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>The single most important quality in a piece of software is simplicity. It&rsquo;s
more important than doing the task you set out to achieve. It&rsquo;s more important
than performance. The reason is straightforward: if your solution is not simple,
it will not be correct or fast.</p>
<p>Given enough time, you&rsquo;ll find that all software which solves sufficiently
complex problems is going to (1) have bugs and (2) have performance problems.
Software with bugs is incorrect. Software with performance problems is not fast.
We will face this fact as surely as we will face death and taxes, and we should
prepare ourselves accordingly. Let&rsquo;s consider correctness first.</p>
<p>Complicated software breaks. Simple software is more easily understood and far
less prone to breaking: there are less moving pieces, less lines of code to keep
in your head, and fewer edge cases. Simple software is more easily tested as
well - after all, fewer code paths to run through. Sure, simple software <em>does</em>
break, and when it does the cause and appropriate solution are often apparent.</p>
<p>Now let&rsquo;s consider performance. You may have some suspicions about your
bottlenecks when you set out, and you should consider them in your approach.
However, when the performance bill comes due, you&rsquo;re more likely to have
overlooked something than not. The only way to find out for sure what&rsquo;s slow is
to measure. Which is easier to profile: a complicated program, or a simple one?
Anyone who&rsquo;s looked at a big enough flame graph knows exactly what I&rsquo;m talking
about.</p>
<p>Perhaps complicated software once solved a problem. That software needs to be
maintained - what is performant and correct today will not be tomorrow. The
workload will increase, or the requirements will change. Software is a living
thing! When you&rsquo;re stressed out at 2 AM on Tuesday morning because the server
shat itself because your 1,831st new customer pushed the billing system over the
edge, do you think you&rsquo;re well equipped to find the problem in a complex piece
of code you last saw a year ago?</p>
<p>When you are faced with these problems, you must seek out the simplest way they
can be solved. This may be difficult to do: perhaps the problem is too large, or
perhaps you were actually considering the solution before considering the
problem. Though difficult it may be, it is your most important job. You need to
take problems apart, identify smaller problems within them and ruthlessly remove
scope until you find the basic problem you can apply a basic solution to. The
complex problem comes later, and it&rsquo;ll be better served by the composition of
simple solutions than with the application of a complex solution.</p>


guid: https://drewdevault.com/2018/07/17/Input-handling-in-wlroots.html
title: Input handling in wlroots
url: https://drewdevault.com/2018/07/17/Input-handling-in-wlroots.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I&rsquo;ve said before that wlroots is a &ldquo;batteries not included&rdquo; kind of library, and
one of the places where that is most apparent is with our approach to input
handling. We implemented a very hands-off design for input, in order to support
many use-cases: desktop input, phones with and without USB-OTG HIDs plugged in,
multiple mice bound to a single cursor, multiple keyboards per seat, simulated
input from fake input devices, on-screen keyboards, input which is processed by
the compositor but not sent to clients&hellip; we support all of these use-cases and
even more. However, the drawback of our powerful design is that it&rsquo;s confusing.
Very confusing.</p>
<p>Let&rsquo;s begin by forgetting about the Wayland part entirely. After all, wlroots is
flexible enough that you can use it without writing a Wayland compositor at all!
It can be used in a similar fashion to tools like GLFW and SDL, to abstract
low-level input (via e.g. libinput) and graphical output (via e.g. DRM). Let&rsquo;s
start here, simply getting input events from wlroots in the first place.</p>
<p>One of the fundamental building blocks of wlroots is the <code>wlr_backend</code>,
which is a resource that abstracts the underlying hardware and exposes a
consistent API for outputs and input devices. Outputs have been discussed
elsewhere, so let&rsquo;s focus just on input devices. Each backend provides an
event: <a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/backend.h#L17"><code>wlr_backend.events.new_input</code></a>. The signal is called with a
reference to a <a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/types/wlr_input_device.h"><code>wlr_input_device</code></a> each time a new input
device appears on the backend - for example, when you plug a mouse into your
computer when using the libinput backend.</p>
<p>The input device can be one of five types, appropriately identified by the
<code>type</code> field. The types are:</p>
<ul>
<li><code>WLR_INPUT_DEVICE_KEYBOARD</code></li>
<li><code>WLR_INPUT_DEVICE_POINTER</code></li>
<li><code>WLR_INPUT_DEVICE_TOUCH</code></li>
<li><code>WLR_INPUT_DEVICE_TABLET_TOOL</code></li>
<li><code>WLR_INPUT_DEVICE_TABLET_PAD</code></li>
</ul>
<p>The type indicates which member of the anonymous union is valid. If
<code>wlr_input_device-&gt;type == WLR_INPUT_DEVICE_KEYBOARD</code>, then
<code>wlr_input_device-&gt;keyboard</code> is a valid pointer to a
<a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/types/wlr_keyboard.h"><code>wlr_keyboard</code></a>.</p>
<p>Let&rsquo;s examine the wlr keyboard more closely now. The keyboard struct also
provides its own events, like <code>key</code> and <code>keymap</code>. If you want to process input
from this keyboard, you need to set up an <a href="https://xkbcommon.org/doc/current/">xkbcommon</a> context for
ingesting the raw scancodes emitted by the <code>key</code> event and converting them to
Unicode and keysyms (e.g. &ldquo;Up&rdquo;) with an XKB keymap. Most of the wlroots examples
<a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/examples/simple.c#L114">implement this</a> if you&rsquo;re looking for a simple reference.</p>
<p>When these events are sent, we just let you process them as you please. They do
not automatically get propagated to any Wayland clients. Communicating these
events to the clients is your responsibility, though we provide you tools to
help - we&rsquo;ll get into that shortly. You don&rsquo;t even have to source the input you
give to Wayland clients from a <code>wlr_input_device</code>, you can just as easily make
them up or get them from the network or anywhere else.</p>
<p>Before we get into details on how to send events to clients, let&rsquo;s examine the
other components in your compositor&rsquo;s input code. First, let&rsquo;s talk about the
cursor.</p>
<p>We provide the <a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/types/wlr_pointer.h"><code>wlr_pointer</code></a> abstraction for getting events from
a &ldquo;pointer&rdquo; device, like a mouse. However, because batteries are not included,
you will find that we only tell you what the pointer device is doing - we don&rsquo;t
act on it. If you want to, for example, display a cursor image <img
src="https://sr.ht/hf39.png" style="display: inline; margin: 0; padding: 0;" />
on screen which moves around when the mouse does, you need to wire this up
yourself. We have tools which can help.</p>
<p>First, let&rsquo;s talk about getting the cursor image to show. You can source the
image from anywhere you want, but you will probably want to leverage
<a href="https://github.com/swaywm/wlroots/blob/master/include/wlr/xcursor.h"><code>wlr_xcursor</code></a>. This is a small wlroots module (forked from the
<code>wayland-cursor</code> library used by Wayland clients) which can read Xcursor themes,
the kind your user will already have installed on their system. Loading up a
cursor theme and getting the pixels from it is pretty straightforward. But what
should you do with those pixels?</p>
<p>Well, now we have to introduce hardware cursors. Many backends support
&ldquo;hardware&rdquo; cursors, which is a feature provided by your low-level graphics stack
(e.g. GPU drivers) for rendering a cursor on the screen. Hardware cursors are
composited by the GPU, which means you can move the cursor around without
re-drawing the things underneath it. This is the most energy- and CPU-efficient
way of drawing your cursor, and you can do it with
<a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/types/wlr_output.h#L179-L190"><code>wlr_output_cursor_set_image</code></a>, specifying which <code>wlr_output</code>
you want it to appear on and at what coordinates. Not all configurations support
hardware cursors, but <code>wlr_output</code> automatically falls back to software cursors
if need be.</p>
<p>Now you have all of the pieces to show a cursor on screen that moves with the
mouse. You can store some X and Y coordinates somewhere, grab an image from an
Xcursor theme, and throw it at your <code>wlr_output</code>, then process input events and
move it around. Then&hellip; you need to consider multiple outputs. And you need to
make sure that it can&rsquo;t be moved outside of an output. And you need to let the
user move it around with a drawing tablet or touch screen as well. And&hellip; well,
it&rsquo;s about to get complicated. That&rsquo;s where our next tool comes in!</p>
<p><a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/types/wlr_cursor.h"><code>wlr_cursor</code></a> is how wlroots saves you from some of this work. It
can display a cursor image on-screen, tie it to multiple input devices,
constrain it to your outputs and move it across multiple displays. It can also
map input from certain devices to certain outputs or regions of the output
layout, change the geometry of inputs from a drawing tablet, and more.</p>
<p>To use <code>wlr_cursor</code>, you should create one (<code>wlr_cursor_create</code>) and as the
backend emits <code>new_input</code> events, bind them to the cursor with
<code>wlr_cursor_attach_input_device</code>. <code>wlr_cursor</code> then raises aggregated events
from all of its devices, which you can catch and handle accordingly - usually
calling a function like <code>wlr_cursor_move</code> and propagating the event to Wayland
clients. You also need to attach a <a href="https://github.com/swaywm/wlroots/blob/master/include/wlr/types/wlr_output_layout.h"><code>wlr_output_layout</code></a> to
the cursor, so it knows how to constrain the cursor movement and can handle
hardware cursors for you.</p>
<p>Aside: the <code>wlr_output_layout</code> module allows you to configure an arrangement of
<code>wlr_output</code>s in physical space. Its function is fairly straightforward and
largely unrelated to our topic - I suggest reading through the header and asking
questions if you need help. Once you make one of these and hand it to a
<code>wlr_cursor</code>, you have a cursor on-screen which moves around when you provide
input and correctly moves throughout a multi-display setup.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> <sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup></p>
<p>Okay, now that we have all of those pieces in place, we can finally start talking
about sending input events to Wayland clients! Before we get into how <em>wlroots</em>
does it, let&rsquo;s talk about how <em>Wayland</em> does it in general.</p>
<p>The top-level resource which manages input for a Wayland client is the
<code>wl_seat</code>. One seat, in rough terms, maps to a single set of input devices used
by a user (a user who is presumably sitting at a seat in front of their
computer). A seat can have up to one keyboard, pointer, touch device, or drawing
tablet each. Each of these devices can then <em>enter</em> or <em>leave</em> any of the
client&rsquo;s surfaces at the compositor&rsquo;s orders.</p>
<p>When you bind to a <code>wl_seat</code>&rsquo;s <code>wl_keyboard</code> and <code>wl_keyboard.enter</code> is raised
on a surface, it means your surface has keyboard focus. The compositor will
follow-up with (or will have already sent) a <code>wl_keyboard.keymap</code> signal to let
you know the layout of this keyboard (e.g. <code>us-intl</code>, <code>de</code>, <code>ru</code>, etc) in the
form of an xkbcommon keymap (the same format we were using with <code>wlr_keyboard</code>
earlier - hint hint). Some number of <code>key</code> and <code>modifier</code> events will likely
follow as the user taps away.</p>
<p>When you bind to a <code>wl_seat</code>&rsquo;s <code>wl_pointer</code> and <code>wl_pointer.enter</code> is raised, it
means a pointer has moved over one of your surfaces. Note that this can be an
entirely separate occasion from receiving keyboard focus. The client is then
expected to provide a cursor image to display (at the moment, Wayland <em>requires</em>
client side cursors. They have to do the whole Xcursor dance we did on the
wlroots side earlier, too. We have some plans to correct this&hellip;). Some number
of <code>motion</code> and <code>button</code> events will likely follow as the user wiggles their
mouse and clicks your windows.</p>
<p>So, how does a wlroots-based compositor facilitate these interactions? With
<a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/types/wlr_seat.h"><code>wlr_seat</code></a>, our abstraction on top of <code>wl_seat</code>. This implements the
whole <code>wl_seat</code> state machine, but again leaves it to you to tweak the knobs as
you wish. You need to decide how your compositor is going to deal with focus -
KDE, Sway, the Librem5 phone UI, an in-vehicle infotainment system; all of these
will have a different approach to focus.</p>
<p>wlroots doesn&rsquo;t render client surfaces for you, and doesn&rsquo;t know where you put
them. Once you figure out where they go, you need to notice when the
<code>wlr_cursor</code> is moved over it and call <code>wlr_seat_pointer_notify_enter</code> with the
pointer&rsquo;s coordinates relative to the surface it entered, along with any
appropriate <code>motion</code> or <code>button</code> events through the relevant <code>wlr_seat</code>
functions. The client will also likely send you a cursor image to display - this
is done with the <code>wlr_seat.events.request_set_cursor</code> event.</p>
<p>When you decide that a surface should receive keyboard focus, call
<code>wlr_seat_keyboard_notify_enter</code>. <code>wlr_seat</code> will automatically handle removing
focus from whatever had it last, and will also grab the keymap and send it to
the client for you, assuming you configured it with <code>wlr_keyboard_set_keymap</code>&hellip;
you did, right? <code>wlr_seat</code> also semi-transparently deals with grabs, the sort of
situation where a client wants to keep keyboard focus for longer than it
normally would, to deal with a context menu or something.</p>
<p>Touch events are similar and should be self-explanatory when you read the
header. Drawing tablet events are a bit different - they&rsquo;re not actually
specified by the core Wayland protocol. Instead, we rig these up with the
<a href="https://cgit.freedesktop.org/wayland/wayland-protocols/tree/unstable/tablet/tablet-unstable-v2.xml">tablet</a> protocol extension and <a href="https://github.com/swaywm/wlroots/blob/7f20ab644347b11fd8242beaf7a6fe42c910d014/include/wlr/types/wlr_tablet_v2.h">wlr_tablet</a>. It
works in much the same way, but you have to explicitly configure it for a
<code>wlr_seat</code> by calling <code>wlr_tablet_create</code> yourself.</p>
<p>So, in short, if you wiggle your mouse, here&rsquo;s what happens:</p>
<ol>
<li>Before you wiggled your mouse, the <code>libinput</code> backend noticed it was plugged
in and raised a <code>new_input</code> event.</li>
<li>Your compositor attached the resulting <code>wlr_pointer</code> to its <code>wlr_cursor</code>,
which it had prepared earlier by looking up an appropriate cursor theme and
letting it know about the display layout.</li>
<li>The <code>wlr_pointer</code> bubbled up a <code>motion</code> event, which was caught by
<code>wlr_cursor</code> and bubbled up to your compositor.</li>
<li>Your compositor called <code>wlr_cursor_move</code> to apply the resulting motion,
constrained by the output layout, which in turn caused the cursor image on
your display to move.</li>
<li>Your compositor then looked around to see if the pointer had moved over any
new surfaces. Since wlroots doesn&rsquo;t handle rendering or know where anything
is displayed, this was a rather introspective question.</li>
<li>You <em>did</em> wiggle it over a new surface, so the compositor called
<code>wlr_seat_notify_pointer_enter</code> after translating the pointer coordinates to
surface-local space. It sent a <code>wlr_seat_notify_pointer_motion</code> for good
measure.</li>
<li>The client noticed the pointer entered it and sent back a cursor image to
show. The compositor was informed of this via
<code>wlr_seat.events.request_set_cursor</code>.</li>
<li>The compositor handled the client&rsquo;s cursor image to <code>wlr_cursor</code>, throwing
away all of that hard work loading up a cursor theme just for a client-side
cursor to come in and ruin it.</li>
</ol>
<p>And there you have it, that&rsquo;s how input works in wlroots. It&rsquo;s really fucking
complicated, isn&rsquo;t it? I think this article puts on display both the incredible
advantages and serious drawbacks of wlroots. Because you have to plug all of
these pieces together yourself, you are afforded an <em>enormous</em> amount of
flexibility. However, you have to do a lot of work and understand a whole lot of
different pieces to get there. Libraries like
<a href="https://github.com/Cloudef/wlc">wlc</a> are much easier to use in this respect,
but if you want to change even a small detail of this process with wlc you are
unable to.</p>
<p>If you have any questions about this article, please reach out to the developers
hanging out in <a href="http://webchat.freenode.net/?channels=sway-devel&amp;uio=d4">#sway-devel on irc.freenode.net</a>. We know this is
confusing, and we&rsquo;re happy to help.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>One more quick note: for multi-DPI setups, you need to provide the <code>wlr_cursor</code> with different cursor images, one for each scale present on the output layout. We have another tool for sourcing Xcursor images at multiple scale factors, check out <a href="https://github.com/swaywm/wlroots/blob/4984ea49eeaa292d66be9e535d93a4d8185f3e18/include/wlr/types/wlr_xcursor_manager.h"><code>wlr_xcursor_manager</code></a>. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Another thing <code>wlr_output_layout</code> is useful for, if you were wondering, is figuring out where to render windows in a multi-output arrangement, where some windows might span multiple outputs. Read the header! <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/07/23/Git-is-already-distributed.html
title: Git is already federated & decentralized
url: https://drewdevault.com/2018/07/23/Git-is-already-distributed.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>There have always been murmurs about &ldquo;replacing GitHub with something
decentralized!&rdquo;, but in the wake of the Microsoft acquisition these murmurs have
become conversations. In particular, this blog post is a direct response to
forge-net (formerly known as <a href="https://github.com/git-federation/gitpub">GitPub</a>). They want to federate and
decentralize git using ActivityPub, the same technology leveraged by Mastodon
and PeerTube. But get this: git is already federated <em>and</em> decentralized!</p>
<p>I already spoke at length about how a large minority of the git community uses
email for collaboration in my <a href="https://drewdevault.com/2018/07/02/Email-driven-git.html">previous article</a> on the subject.
Definitely give it a read if you haven&rsquo;t already. In this article I want to
focus on comparing this model with the possibilities afforded by ActivityPub
and provide direction for new forge<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> projects to work towards embracing and
improving git&rsquo;s email-based collaboration tools.</p>
<p>The main issue with using ActivityPub for decentralized git forges boils down to
email simply being a better choice. The advantages of email are numerous. It&rsquo;s
already standardized and has countless open source implementations, many in the
standard libraries of almost every programming language. It&rsquo;s decentralized and
federated, and it&rsquo;s <em>already</em> integrated with git. Has been since day one!  I
don&rsquo;t think that we should replace web forges with our email clients, not at
all. Instead, web forges should embrace email to communicate with each other.</p>
<p>Let me give an example of how this could play out. On my platform,
<a href="https://meta.sr.ht">sr.ht</a>, users can view their git repositories on the web
(duh). One of my goals is to add some UI features here which let them select a
range of commits and prepare a patchset for submission via <a href="https://www.git-scm.com/docs/git-send-email">git
send-email</a>. They&rsquo;ll enter an email address (or addresses) to
send the patch(es) to, and we&rsquo;ll send it along on their behalf.  This email
address might be a mailing list on another sr.ht instance in the wild! If so,
the email gets recognized as a patch and displayed on the web with a pretty diff
and code review tools. Inline comments automatically get formatted as an email
response. This shows up in the user&rsquo;s inbox and sr.ht gets copied on it, showing
it on the web again.</p>
<p>I think that workflow looks an awful lot like the workflow forge-net hopes to
realize! Here&rsquo;s where it gets good, though. What if the emails the user puts in
are <code>linux-kernel@vger.kernel.org</code> and a handful of kernel maintainers? Now your
git forge can suddenly be used to contribute to the Linux kernel! ActivityPub
would build a <em>second</em>, incompatible federation of projects, while ignoring the
already productive federation which powers many of our most important open
source projects.</p>
<p>git over email is already supported by a tremendous amount of open source
software. There&rsquo;s tools like <a href="https://www.gnu.org/software/mailman/">mailman</a> which provide mailing lists and
public archives, or <a href="https://public-inbox.org/">public-inbox</a>, which archives email in git,
or <a href="http://jk.ozlabs.org/projects/patchwork/">patchworks</a> for facilitating code review over email. Some email
clients have grown features which make them more suitable for git, such as
<a href="http://mutt.org">mutt</a>. These are the nuts and bolts of hundreds of important projects,
including Linux, *BSD, gcc, Clang, postgresql, MariaDb, emacs, vim, ffmpeg,
Linux distributions like Debian, Fedora, Arch, Alpine, and countless other
projects, including git itself! These projects are incredibly important,
foundational projects upon which our open source empire is built, and the tools
they use already provide an open, federated protocol for us to talk to.</p>
<p>Not only is email <em>better</em>, but it&rsquo;s also <em>easier</em> to implement. Programming
tools for email are very mature. I recently started experimenting with building
an ActivityPub service, and it was crazy difficult. I had to write a whole lot
of boilerplate and understand new and still-evolving specifications, not to
mention setting up a public-facing server with a domain and HTTPs to test
federation with other implementations. Email is comparatively easy, it&rsquo;s built
into the standard library. You can shell out to git and feed the patch to the
nearest SMTP library in only a handful of lines of code. I bet every single
person who reads this article already has an email address, so the setup time
approaches zero.</p>
<p>Email also puts the power in the hands of the user right away. On Mastodon there
are occasional problems of instance owners tearing down their instance on short
notice, taking with them all of their user&rsquo;s data. If everything is being
conducted over email instead, all of the data already lives in the user&rsquo;s inbox.
Freely available tools can take their mail spool and publish a new archive if
our services go down. Mail archives can be trivially made redundant across many
services. This stuff is seriously resilient to failure. Email was designed when
networks were measured in bits per second and often connected through a single
unreliable route!</p>
<p>I&rsquo;m not suggesting that the approach these projects use for collaboration is
perfect. I&rsquo;m suggesting that we should embrace it and solve these problems
instead of throwing out the baby with the bathwater. Tools like <code>git send-email</code>
can be confusing at first, which is why we should build tools like web forges
that smooth over the process for novices, and write better docs to introduce
people to the tools (I recently <a href="https://man.sr.ht/git.sr.ht/send-email.md">wrote a guide</a> for sr.ht users).</p>
<p>Additionally, many popular email clients have bastardized email to the point
where the only way to use git+email for many people starts with abandoning the
email client they&rsquo;re used to using. This can also be solved by having forges
send the emails for them, and process the replies. We can also support open
source mail clients by building better tools to integrate our emails with them.
Setting up the mail servers on the other end can be difficult, too, but we
should invest in better mail server software, something which would definitely
be valuable even setting aside the matter of project forges.</p>
<p>We need to figure out something for bugs as well, perhaps based on Debian&rsquo;s work
on <a href="https://www.debian.org/Bugs/">Debbugs</a>. Other areas of development, such as
continuous integration, I find are less difficult problems. Many build services
already support sending the build results by email, we just need to find a way
to get our patches to them (something I&rsquo;m working on with sr.ht). But we should
take these problems one step at a time. Let&rsquo;s focus on improving the patch
workflow git endorses, and as our solutions shake out the best solutions to our
other problems will become more and more apparent.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Forge refers to any software which provides comprehensive tools for project hosting. This originally referred to SourceForge but is now a category of software which includes GitHub, BitBucket, GitLab, Gogs/Gitea, etc. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/07/29/Wayland-shells.html
title: Writing a Wayland compositor with wlroots: shells
url: https://drewdevault.com/2018/07/29/Wayland-shells.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I apologise for not writing about wlroots more frequently. I don&rsquo;t really enjoy
working on the McWayface codebase this series of blog posts was originally
about, so we&rsquo;re just going to dismiss that and talk about the various pieces of
a Wayland compositor in a more free-form style. I hope you still find it useful!</p>
<p>Today, we&rsquo;re going to talk about shells. But to make sure we&rsquo;re on the same page
first, a quick refresher on surfaces. A basic primitive of the Wayland protocol
is the concept of a &ldquo;surface&rdquo;. A surface is a rectangular box of pixels sent
from the client to the compositor to display on-screen. A surface can source
its pixels from a number of places, including raw pixel data in memory, or
opaque handles to GPU resources that can be rendered without copying pixels on
the CPU. These surfaces can also evolve over time, using &ldquo;damage&rdquo; to indicate
which parts have changed to reduce the workload of the compositor when
re-rendering them. However, making a surface and filling it with pixels is not
enough to get the compositor to show them.</p>
<p>Shells are how surfaces in Wayland are given meaning. Consider that there are
several kinds of surfaces you&rsquo;ll encounter on your desktop. There are
application windows, sure, but there are also tooltips, right-click menus and
menubars, desktop panels, wallpapers, lock screens, on-screen keyboards, and so
on. Each of these has different semantics - your wallpaper cannot be minimized
or dragged around and resized, but your application windows can be.  Likewise,
your application windows cannot cover the entire screen and soak up all input
like your lock screen can. Each of these use cases is fulfilled with a <em>shell</em>,
which generally takes a surface resource, assigns it a role (e.g.  application
window), and returns a handle with shell-specific interfaces for manipulating
it.</p>
<h2 id="shells-in-wlroots">Shells in wlroots</h2>
<p>I want to first discuss features common to shells as implemented by wlroots.
Each shell has a shell-specific interface that sits on top of the surface. Each
time a client connects and creates one of these, the shell raises a <code>wl_signal</code>,
<code>events.new_surface</code>, and passes to it a pointer to a shell-specific structure
which encapsulates that shell surface&rsquo;s state.</p>
<p>Many shells require some configuration between the creation of the shell surface
and displaying it on screen. For example, during this period application windows
will typically set the window title so that the compositor never has to show an
empty title. All Wayland interfaces aim for atomicity, so that all changes are
applied in a single fell swoop and we never display an invalid frame. This is
why Wayland is known for addressing vsync problems X suffers from, but is
pervasive across the ecosystem. Even things like setting the window title are
done atomically.</p>
<p>So, once the client is done communiciating the new shell surface&rsquo;s desired
traits to the compositor, it will commit the surface to atomically apply the
changes. The first time this happens, the client is ready to be shown, and the
shell-specific wlroots shell surface interface will communicate this to you with
the surface&rsquo;s <code>events.map</code> signal. The reverse is sometimes communicated with
<code>events.unmap</code>, when the shell surface should be hidden.</p>
<h2 id="xdg-shell">xdg-shell</h2>
<p>xdg-shell is currently the only shell whose protocol is considered stable, and
it is the shell which describes application windows. You can read the xdg-shell
protocol specification (XML)
<a href="https://cgit.freedesktop.org/wayland/wayland-protocols/tree/stable/xdg-shell/xdg-shell.xml">here</a>
(you are strongly encouraged to read through the XML for all protocols mentioned
in this article).</p>
<p>The xdg-shell is quite complicated, as it attempts to encapsulate every feature
of a typical graphical desktop session in a single protocol. An xdg-shell
surface is a <code>wl_surface</code> wrapped twice - once in a <code>xdg_surface</code> and then again
in a <code>xdg_toplevel</code> or <code>xdg_popup</code>, depending on what kind of window it is. The
wlroots <code>wlr_xdg_surface</code> type (the one emitted by
<code>xdg_shell.events.new_surface</code>) contains tagged union of <code>wlr_xdg_toplevel</code> and
<code>wlr_xdg_popup</code>, selected from the <code>role</code> field. You can wire up the xdg-shell
with <code>wlr_xdg_shell_create</code>.</p>
<p>Most application windows you see are called toplevels. These windows are the
root node of a tree of surfaces which may include arbitrarily nested popups, for
example, as you navigate through a deep menu. These windows can have titles;
parent surfaces; app IDs (e.g. &ldquo;gnome-calculator&rdquo;); minimum and maximum sizes;
and maximized, minimized, and fullscreen states. They also often<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> draw their
own window decorations and drop shadows, and tell the compositor when you click
and drag on the titlebar to move or resize the window.  Unfortunately, if the
client is not responding or misbehaving, the user cannot use these controls to
move, resize, or minimize the window<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>.</p>
<p>The compositor can tell the window to adopt a specific size, though the client
can choose to ignore this. The compositor also lets the client know when it&rsquo;s
&ldquo;activated&rdquo;, which is used by GTK+, for example, to start rendering the caret
and render a different set of client-side decorations. It can also toggle the
fullscreen, minimized, maximized, and other states.</p>
<p>Each of the various state transitions involved are expressed through
the <code>wlr_xdg_toplevel.events</code> signals. The most recent atomically agreed-upon
state is stored in <code>wlr_xdg_toplevel.current</code>. When each of the signals in
<code>events</code> are emitted, the state change will have been applied to
<code>client_pending</code>. However, you must consent to these changes by calling a
corresponding function on the xdg_toplevel (e.g.
<code>wlr_xdg_toplevel_set_fullscreen</code>), which will apply the change to
<code>server_pending</code>. You shouldn&rsquo;t consider these changes atomically set until the
<code>wlr_surface.events.commit</code> signal has been raised. At that point, you can start
showing the window in fullscreen or whatever. There&rsquo;s also some
configure/ack-configure stuff going on here which may eventually become relevant
to you<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>, but wlroots takes care it for the most part.</p>
<p>The popup interface is used to show a &ldquo;popup&rdquo; window, which can be used for a
variety of purposes. These include context menus (or &ldquo;right click&rdquo; menus),
tooltips, some confirmation modals<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>, etc. The lifecycle of a popup resource
is managed similarly to that of a toplevel resource, of course with different
states that can be atomically updated. Arguably, the most fundamental of these
states is the relative X and Y position of the popup with respect to its parent
toplevel surface.</p>
<p>The position of the popup can be influenced by an extraordinarily complicated
interface called <code>xdg_positioner</code>, also provided by xdg-shell. Since these
articles focus on the compositor side of things, and they focus on using
wlroots, I can thankfully save you from understanding most of the specifics of
this interface. The purpose of this interface is to adjust the position and size
of <code>xdg_popup</code> surfaces with respect to the display they live on - for example,
to prevent them from being partially off-screen. The rub is that if you&rsquo;re using
wlroots, when the popup is created you can just call
<code>wlr_xdg_popup_unconstrain_from_box</code> to deal with everything, passing it a box
which represents the available space surrounding the parent toplevel for the
popup to be placed in.</p>
<p>Popups are also able to take &ldquo;grabs&rdquo;, which indicate that they should keep
focus without respect to any of the other goings-on of the seat. This is used so
that you can, for example, use the keyboard to pick items from a context menu.
Grabs are automatically handled for you with <code>wlr_seat</code> for you. If you want to
deny or cancel grabs, you can do so through the appropriate <code>wlr_seat</code>
interfaces.</p>
<p>One last note: xdg-shell only recently became stable, so client support for the
stable version is hit and miss. The last unstable protocol, xdg-shell v6, is
also supported by wlroots. It mostly behaves in the same way. Eventually it
will be removed from wlroots.</p>
<h2 id="layer-shell">layer-shell</h2>
<p>Under the umbrella of wlroots, 8 Wayland compositors have been collaborating on
the design of a new shell for desktop shell components. The result is <a href="https://github.com/swaywm/wlr-protocols/blob/master/unstable/wlr-layer-shell-unstable-v1.xml">layer
shell
(XML)</a>.
The purpose of this shell is to provide an interface for desktop components like
panels, lock screens, wallpapers, on-screen keyboards, notifications, and so on,
to display on your compositor.</p>
<p>The layer-shell is organized into four discrete layers: background, bottom, top,
and overlay, which are rendered in that order. Between bottom and top,
application windows are displayed. A wallpaper client might choose to go in the
bottom layer, while a notification could show on the top layer, and a panel on
the bottom layer.</p>
<p>The compositor&rsquo;s job is to decide where to place each surface and how large the
surface can be. The client can specify either or both of its dimensions (width
and height) for the compositor to specify, then provide some hints for the
compositor to do so. The client can, for example, choose to be anchored to edges
of the screen. A notification might be anchored to <code>TOP | RIGHT</code>, and a panel
might be anchored to <code>LEFT | BOTTOM | RIGHT</code>. A layer surface anchored to an
edge, like our panel, can also request an exclusive zone, which is a number of
pixels from the edge that should not be occluded by other layer surfaces or
application windows. This is used, for example, when maximizing application
windows to prevent them from occluding the panel (or in sway&rsquo;s case, when
arranging tiled windows).</p>
<p>Layer surfaces also have special keyboard input semantics. Some layer surfaces
want to receive keyboard input, such as an application launcher overlay. Others
might prefer that application windows continue to receive keyboard events, such
as a notification. To this end, a layer surface can toggle a boolean indicating
its &ldquo;keyboard interactivity&rdquo;. For layers beneath application windows, layer
surfaces participate in keyboard focus normally, usually meaning they need to be
clicked to receive keyboard focus. Above application windows, the top-most layer
always has keyboard focus if it requests it.</p>
<p>In wlroots, you can wire up a layer shell to the display with
<code>wlr_layer_shell_create</code>. From there it behaves similarly to xdg-shell with
respect to the creation of new surfaces and the handling of atomic state. The
main concern of yours is that, when the surface is committed, you need to
arrange the surfaces in the affected layer and communicate the final dimensions
of the layer surface to the client with <code>wlr_layer_surface_configure</code>. You can
implement the arrangement however you want, but you may find the <a href="https://github.com/swaywm/sway/blob/master/sway/desktop/layer_shell.c#L18-L215">sway
implementation</a> to be a useful reference. Also check out the
wlroots <a href="https://github.com/swaywm/wlroots/blob/master/examples/layer-shell.c">example client</a> to test out your implementation.</p>
<p>Layer surfaces can also have popups, for example when right-clicking on a
taskbar. This borrows xdg-shell&rsquo;s xdg_popup interface, except the parent is set
to the layer surface (this is explicitly allowed for through the xdg_popup spec,
and you may see future shells doing something similar). Most of your code for
xdg_popups can be reused with layer surfaces.</p>
<h2 id="xwayland">Xwayland</h2>
<p>Some Wayland developers turn up their nose when I refer to Xwayland as a shell,
and perhaps with good reason. However, wlroots treats Xwayland like a shell, so
the API remains consistent. For that reason, we&rsquo;ll treat it as one in this
article as well.</p>
<p>We figured that you might be writing a Wayland compositor so that you <em>don&rsquo;t</em>
have to write an X11 window manager, too. So we wrote one for you, and it&rsquo;s
called <code>wlr_xwayland</code>. This interface provides an abstraction over Xwayland
which makes it behave similarly to our other shells. It still lets you dig your
heels into it in any degree so that you can adjust the behavior of your
compositor to suit X-specific needs as necessary.</p>
<p>The resulting wlr_xwayland API is similar to the other shells we&rsquo;ve described.
We have a series of events for configuring Xwayland surfaces, a map and unmap
event, and we expose a whole bunch of info about Xwayland surfaces so you can
make the judgement call about how much or how little to obey their requests (X11
windows make more unreasonable requests than other shells, since X11 was the
wild wild west and a lot of clients took advantage of that).</p>
<p>This should be enough to get you started, and if you have questions ask on IRC
for the time being. I could go into more detail, but I think Xwayland deserves
its own article, and probably not written by me.</p>
<h2 id="other-shells">Other shells</h2>
<p>There are three other shells of note. Two are not very interesting:</p>
<ul>
<li>wl_shell, the now-deprecated original desktop shell of Wayland</li>
<li>ivi-shell, used for &ldquo;in-vehicle infotainment&rdquo; systems running Wayland</li>
</ul>
<p>wlroots supports neither (though I guess we&rsquo;d accept a patch adding IVI-shell
support, maybe if the vehicle industry was open to improving that protocol&hellip;),
and neither is interesting for desktops, phones, etc. You probably don&rsquo;t need to
worry about them.</p>
<p>The other is the fullscreen-shell, which is used for optimizing the rendering of
fullscreen appliations. I don&rsquo;t know much about how it works, and it&rsquo;s not
supported by wlroots yet; it&rsquo;s not required of a functional Wayland compositor.
Maybe someday!</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p><a href="https://cgit.freedesktop.org/wayland/wayland-protocols/tree/unstable/xdg-decoration/xdg-decoration-unstable-v1.xml">But not always</a>. You&rsquo;re welcome. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Which is one of the reasons we made the protocol mentioned in footnote<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>For example, this is relevant for sway, which needs to reach deeper into our shell implementations to atomically syncronize the resizing of several clients at once when rearranging the layout. <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>Some popup windows, the GTK+ file chooser for example, prefer to make a new xdg_toplevel and assign its parent to the application window. This is useful if you want your window to show up in taskbars, be able to be minimized and maximized separately, etc. <a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/08/05/Local-mail-server.html
title: Setting up a local dev mail server
url: https://drewdevault.com/2018/08/05/Local-mail-server.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>As part of my work on <a href="https://meta.sr.ht">lists.sr.ht</a>, it was necessary for
me to configure a self-contained mail system on localhost that I could test
with. I hope that others will go through a similar process in the future when
they set up <a href="https://git.sr.ht/~sircmpwn/lists.sr.ht">the code</a> for hacking on
locally or when working on other email related software, so here&rsquo;s a guide on
how you can set it up.</p>
<p>There are lots of things you can set up on a mail server, like virtual mail
accounts backed by a relational database, IMAP access, spam filtering, and so
on. We&rsquo;re not going to do any of that in this article - we&rsquo;re just interested in
something we can test our email code with. To start, install your distribution
of <code>postfix</code> and pop open that <code>/etc/postfix/main.cf</code> file.</p>
<p>Let&rsquo;s quickly touch on the less interesting config keys to change. If you want
the details about how these work, consult the postfix manual.</p>
<ul>
<li><em>myhostname</em> should be your local hostname</li>
<li><em>mydomain</em> should also be your local hostname</li>
<li><em>mydestination</em> should be <code>$myhostname, localhost.$mydomain, localhost</code></li>
<li><em>mynetworks</em> should be <code>127.0.0.0/8</code></li>
<li><em>home_mailbox</em> should be <code>Maildir/</code></li>
</ul>
<p>Also ensure your hostname is set up right in <code>/etc/hosts</code>, something like this:</p>
<pre><code>127.0.0.1 homura.localdomain homura
</code></pre><p>Okay, those are the easy ones. That just makes it so that your mail server
oversees mail delivery for the <code>127.0.0.0/8</code> network (localhost) and delivers
mail to local Unix user mailboxes. It will store incoming email in each user&rsquo;s
home directory at <code>~/Maildir</code>, and will deliver email to other Unix users. Let&rsquo;s
set up an email client for reading these emails with. Here&rsquo;s my development
<a href="http://mutt.org">mutt</a> config:</p>
<pre><code>set edit_headers=yes
set realname=&quot;Drew DeVault&quot;
set from=&quot;sircmpwn@homura&quot;
set editor=vim
set spoolfile=&quot;~/Maildir/&quot;
set folder=&quot;~/Maildir/&quot;
set timeout=5
color index blue default ~P
</code></pre><p>Make any necessary edits. If you use mutt to read your normal mail, I suggest
also setting up an alias which runs <code>mutt -C path/to/dev/config</code>. Now, you
should be able to send an email to yourself or other Unix accounts with
mutt<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. Hooray!</p>
<p>To accept email over SMTP, mozy on over to <code>/etc/postfix/master.cf</code> and
uncomment the submission service. You&rsquo;re looking for something like this:</p>
<pre><code>127.0.0.1:submission inet n       -       n       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_tls_auth_only=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit
#  -o milter_macro_daemon_name=ORIGINATING
</code></pre><p>This will permit delivery via localhost on the submission port (587) to anyone
whose hostname is in <code>$mydestination</code>. A good old <code>postfix reload</code> later and you
should be able to send yourself an email with SMTP:</p>
<pre><code>$ telnet 127.0.0.1 587
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
220 homura ESMTP Postfix
EHLO example.org
250-homura
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 SMTPUTF8
MAIL FROM:&lt;sircmpwn@homura&gt;
250 2.1.0 Ok
RCPT TO:&lt;sircmpwn@homura&gt; 
250 2.1.5 Ok
DATA
354 End data with &lt;CR&gt;&lt;LF&gt;.&lt;CR&gt;&lt;LF&gt;
From: Drew DeVault &lt;sircmpwn@homura&gt;
To: Drew DeVault &lt;sircmpwn@homura&gt;
Subject: Hello world

Hey there 
.
250 2.0.0 Ok: queued as 8267416366B
QUIT
221 2.0.0 Bye
Connection closed by foreign host.
</code></pre><p>Pull up mutt again to read this. Any software which will be sending out mail and
speaks SMTP (for example, sr.ht) can be configured now. Last step is to set up
LTMP delivery to lists.sr.ht or any other software you want to process incoming
emails. I want most mail to deliver normally - I only want LTMP configured for
my lists.sr.ht test domain. I&rsquo;ll set up some transport maps for this purpose. In
<code>main.cf</code>:</p>
<pre><code>local_transport = local:$myhostname
transport_maps = hash:/etc/postfix/transport
</code></pre><p>Then I&rsquo;ll edit <code>/etc/postfix/transport</code> and add these lines:</p>
<pre><code>lists.homura.localdomain lmtp:unix:/tmp/lists.sr.ht-lmtp.sock
homura.localdomain local:homura
</code></pre><p>This will deliver mail normally to <code>$user@homura</code> (my hostname), but will
forward mail sent to <code>$user@lists.homura</code> to the Unix socket where the
<a href="https://git.sr.ht/~sircmpwn/lists.sr.ht/tree/master/listssrht-lmtp">lists.sr.ht LMTP
server</a> lives.</p>
<p>Add the subdomain to <code>/etc/hosts</code>:</p>
<pre><code>127.0.0.1 lists.homura.localdomain lists.homura
</code></pre><p>Run <code>postmap /etc/postfix/transport</code> and <code>postfix reload</code> and you&rsquo;re good to go.
If you have the lists.sr.ht daemon working, send some emails to
<code>~someone/example-list@lists.$hostname</code> and you should see them get picked up.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Mutt crash course: run <code>mutt</code>, press <code>m</code> to compose a new email, enter the recipient (<code>$USER@$HOSTNAME</code> to send to yourself) and the subject, then compose your email, exit the editor, and press <code>y</code> to send. A few moments later the email should arrive. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/08/08/Signal.html
title: I don't trust Signal
url: https://drewdevault.com/2018/08/08/Signal.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Occasionally when Signal is in the press and getting a lot of favorable
discussion, I feel the need to step into various forums, IRC channels, and so
on, and explain why I don&rsquo;t trust Signal. Let&rsquo;s do a blog post instead.</p>
<p>Off the bat, let me explain that I expect a tool which claims to be secure to
actually be secure. I don&rsquo;t view &ldquo;but that makes it harder for the average
person&rdquo; as an acceptable excuse. If Edward Snowden and Bruce Schneier are going
to spout the virtues of the app, I expect it to <em>actually</em> be secure when it
matters - when vulnerable people using it to encrypt sensitive communications
are targeted by smart and powerful adversaries.</p>
<p>Making promises about security without explaining the tradeoffs you made in
order to appeal to the average user is unethical. Tradeoffs are necessary - but
self-serving tradeoffs are not, and it&rsquo;s your responsibility to clearly explain
the drawbacks and advantages of the tradeoffs you make. If you make broad and
inaccurate statements about your communications product being &ldquo;secure&rdquo;, then
when the political prisoners who believed you are being tortured and hanged,
it&rsquo;s on you. The stakes are serious. Let me explain why I don&rsquo;t think Signal
takes them seriously.</p>
<h2 id="google-play">Google Play</h2>
<p>Why do I make a big deal out of Google Play and Google Play Services? Well, some
people might trust Google, the company. But up against nation states, it&rsquo;s no
contest - Google has ties to the NSA, has been served secret subpoenas, and is
literally the world&rsquo;s largest machine designed for harvesting and analyzing
private information about their users. Here&rsquo;s what Google Play Services
<em>actually</em> is: <strong>a rootkit</strong>. Google Play Services lets Google do silent
background updates on apps on your phone and give them any permission they want.
Having Google Play Services on your phone means your phone is not secure.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p>
<p>For the longest time, Signal wouldn&rsquo;t work without Google Play Services, but
Moxie (the founder of Open Whisper Systems and maintainer of Signal) finally
fixed this in 2017. There was also a long time when Signal was only available on
the Google Play Store. Today, you can <a href="https://signal.org/android/apk/">download the APK directly from
signal.org</a>, but&hellip; well, we&rsquo;ll
get to that in a minute.</p>
<h2 id="f-droid">F-Droid</h2>
<p>There&rsquo;s an alternative to the Play Store for Android.
<a href="https://f-droid.org">F-Droid</a> is an open source app &ldquo;store&rdquo; (repository would
be a better term here) which only includes open source apps (which Signal
thankfully is).  By no means does Signal have to <em>only</em> be distributed through
F-Droid - it&rsquo;s certainly a compelling alternative. This has been proposed, and
Moxie has <a href="https://github.com/signalapp/Signal-Android/issues/127">definitively shut the discussion
down</a>. Admittedly this
is from 2013, but his points and the arguments against them haven&rsquo;t changed. Let
me quote some of his positions and my rebuttals:</p>
<blockquote>
<p>No upgrade channel. Timely and automatic updates are perhaps the most
effective security feature we could ask for, and not having them would be a
real blow for the project.</p>
</blockquote>
<p>F-Droid supports updates. If you&rsquo;re concerned about moving your updates quickly
through the (minimal) bureaucracy of F-Droid, you can always run your own
repository. Maybe this is a lot of work?<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> I wonder how the workload compares
to <a href="https://signal.org/blog/signal-and-giphy-update/">animated gif search</a>, a very important feature for security
concious users. I bet that <a href="https://signal.org/blog/signal-foundation/">50 million dollar donation</a> could help,
given how many people operate F-Droid repositories on a budget of $0.</p>
<blockquote>
<p>No app scanning. The nice thing about market is the server-side APK scanning
and signature validation they do. If you start distributing APKs around the
internet, it&rsquo;s a reversion back to the PC security model and all of the
malware problems that came with it.</p>
</blockquote>
<p>Try searching the Google Play Store for &ldquo;flashlight&rdquo; and look at the permissions
of the top 5 apps that come up. All of them are harvesting and selling the
personal information of their users to advertisers. Is this some kind of joke?
F-Droid is a curated repository, like Linux distributions. Google Play is a
malware distributor.  Packages on F-Droid are reviewed by a human being and are
<a href="https://f-droid.org/en/docs/Signing_Process/">cryptographically signed</a>. If you
run your own F-Droid repo this is even less of a concern.</p>
<p>I&rsquo;m not going to address all of Moxie&rsquo;s points here, because there&rsquo;s a deeper
problem to consider. I&rsquo;ll get into more detail shortly. You can read the
6-year-old threads tearing Moxie&rsquo;s arguments apart over and over again until
GitHub added the feature to lock threads, if you want to see a more in-depth
rebuttal.</p>
<h2 id="the-apk-direct-download">The APK direct download</h2>
<p>Last year Moxie added an official APK download to signal.org. He said this was
up for &ldquo;<a href="https://github.com/signalapp/Signal-Android/issues/127#issuecomment-286223680">harm reduction</a>&rdquo;, to avoid people using unofficial
builds they find around the net. The download page is covered in warnings
telling you that it&rsquo;s for advanced users only, it&rsquo;s insecure, would you please
go to the Google Play store you stupid user. I wonder, has Moxie considered
communicating to people the risks of using the Google Play version?<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup></p>
<p>The APK direct download doesn&rsquo;t even accomplish the stated goal of &ldquo;harm
reduction&rdquo;. The user has to manually verify the checksum, and figure out how to
do it on a phone, no less. A checksum isn&rsquo;t a signature, by the way - if your
government- or workplace- or abusive-spouse-installed certificate authority gets
in the way they can replace the APK <em>and</em> its checksum with whatever they want.
The app has to update itself, using a similarly insecure mechanism. F-Droid
handles updates and actually signs their packages. This is a no brainer, Moxie,
why haven&rsquo;t you put Signal on F-Droid yet?</p>
<h2 id="why-is-signal-like-this">Why is Signal like this?</h2>
<p>So if you don&rsquo;t like all of this, if you don&rsquo;t like how Moxie approaches these
issues, if you want to use something else, what do you do?</p>
<p>Moxie knows about everything I&rsquo;ve said in this article. He&rsquo;s a very smart guy
and I am under no illusions that he doesn&rsquo;t understand everything I&rsquo;ve put
forth. I don&rsquo;t think that Moxie makes these choices because he thinks they&rsquo;re
the right thing to do. He makes arguments which don&rsquo;t hold up, derails threads,
leans on logical fallacies, and loops back around to long-debunked positions
when he runs out of ideas. I think this is deliberate. An open source software
team reads this article as a list of things they can improve on and gets
started. Moxie reads this and prepares for war. Moxie can&rsquo;t come out and
say it openly, but he&rsquo;s made the decisions he has made because they serve his
own interests.</p>
<p>Lots of organizations which are pretending they don&rsquo;t make self-serving decisions at
their customer&rsquo;s expense rely on argumentative strategies like Moxie does. If
you can put together an argument which on the surface appears reasonable, but
requires in-depth discussion to debunk, passerby will be reassured that your
position is correct, and that the dissenters are just trolls. They won&rsquo;t have
time to read the lengthy discussion which demonstrates that your conclusions
are wrong, especially if you draw the discussion out like Moxie does. It can be
hard to distinguish these from genuine positions held by the person you&rsquo;re
talking to, but when it conveniently allows them to make self-serving plays,
it&rsquo;s a big red flag.</p>
<p>This is a strong accusation, I know. The thing which convinced me of its truth
is Signal&rsquo;s centralized design and hostile attitude towards forks. In open
source, when a project is making decisions and running things in a way you don&rsquo;t
like, you can always fork the project. This is one of the fundamental rights
granted to you by open source. It has a side effect Moxie doesn&rsquo;t want, however.
It reduces his power over the project. Moxie has a clever solution to this:
centralized servers and trademarks.</p>
<h2 id="trust-federation-and-peer-to-peer-chat">Trust, federation, and peer-to-peer chat</h2>
<p>Truly secure systems do not require you to trust the service provider. This is
the point of end-to-end encryption. But we have to trust that Moxie is running
the server software he says he is. We have to trust that he isn&rsquo;t writing down a
list of people we&rsquo;ve talked to, when, and how often. We have to trust not only
that Moxie is trustworthy, but given that Open Whisper Systems is based in San
Francisco we have to trust that he hasn&rsquo;t received a national security letter,
too (by the way, Signal doesn&rsquo;t have a warrant canary). Moxie can <em>tell</em> us he
doesn&rsquo;t store these things, but he could. <strong>Truly secure systems don&rsquo;t require
trust</strong>.</p>
<p>There are a couple of ways to solve this problem, which can be used in tandem.
We can stop Signal from knowing when we&rsquo;re talking to each other by using
peer-to-peer chats. This has some significant drawbacks, namely that both users
have to be online at the same time for their messages to be delivered to each
other. You can still fall back to peer-to-server-to-peer when one peer is
offline, however. But this isn&rsquo;t the most important of the two solutions.</p>
<p>The most important change is federation. Federated services are like email, in
that Alice can send an email from gmail.com to Bob&rsquo;s yahoo.com address. I should
be able to stand up a Signal server, on my own hardware where I am in control of
the logs, and communicate freely with other Signal servers, including Open
Whisper&rsquo;s servers. This distributes the security risks across hundreds of
operators in many countries with various data extradition laws. This turns what
would today be easy for the United States government to break and makes it much,
much more difficult. Federation would also open the possibility for bridging the
gap with several other open source secure chat platforms to all talk on the same
federated network - which would spur competition and be a great move for users
of all chat platforms.</p>
<p>Moxie forbids you from distributing branded builds of the Signal app, and if you
rebrand he forbids you from using the official Open Whisper servers. Because his
servers don&rsquo;t federate, that means that users of Signal forks <em>cannot talk to
Signal users</em>. This is a truly genius move. No fork of Signal<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup> to date has
ever gained any traction, and never will, because you can&rsquo;t talk to any Signal
users with them. In fact, there are no third-party applications which can
interact with Signal users in any way. Moxie can write as many blog posts which
appeal to wispy ideals and &ldquo;moving ecosystems&rdquo; as he wants<sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup>, but those are
all <em>really</em> convenient excuses for an argument which allows him to design
systems which serve his own interests.</p>
<p>No doubt these are non-trivial problems to solve. But I have <em>personally</em> been
involved in open source projects which have collectively solved similarly
difficult problems a thousand times over with a combined budget on the order of
tens of thousands of dollars.</p>
<p>What were you going to do with that 50 million dollars again?</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>&ldquo;But how is AOSP any better?&rdquo; This is a common strawman counter-argument. Fact: There is empirical evidence which shows that Google Play Services does silent updates and can obtain any permission on your phone: a rootkit. There is no empirical evidence to suggest AOSP has similar functionality. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>No, it&rsquo;s not. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>Probably not, because that wouldn&rsquo;t be self-serving. But I&rsquo;m getting ahead of myself. <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>See <a href="https://github.com/LibreSignal/LibreSignal">LibreSignal</a> and <a href="https://github.com/SilenceIM/Silence#silence-">Silence</a>, particularly <a href="https://github.com/LibreSignal/LibreSignal/issues/37#issuecomment-217211165">this thread</a>. <a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:5" role="doc-endnote">
<p>See <a href="https://signal.org/blog/the-ecosystem-is-moving/">Reflections: The ecosystem is moving</a>. Yes, that&rsquo;s the unedited title. <a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/08/22/Commons-clause-will-destroy-open-source.html
title: The Commons Clause will destroy open source
url: https://drewdevault.com/2018/08/22/Commons-clause-will-destroy-open-source.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>An alarmist title, I know, but it&rsquo;s true. If the <a href="https://commonsclause.com/">Commons
clause</a> were to be adopted by all open source
projects, they would cease to be open source<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>, and therefore the Commons
clause is trying to destroy open source. When this first appeared I spoke out
about it in discussion threads around the net, but didn&rsquo;t think anyone would
take it seriously. Well, yesterday, some parts of Redis <a href="https://redislabs.com/community/commons-clause/">became proprietary
software</a>.</p>
<p>The Commons Clause promoted by Kevin Wang presents one of the greatest
existential threats to open source I&rsquo;ve ever seen. It preys on a vulnerability
open source maintainers all suffer from, and one I can strongly relate to. It
<em>sucks</em> to not be able to make money from your open source work. It <em>really</em>
sucks when companies are using your work to make money for themselves. If a
solution presents itself, it&rsquo;s tempting to jump at it. But the Commons Clause
doesn&rsquo;t present a solution for supporting open source software. It presents a
framework for turning open source software into proprietary software.</p>
<p>What should we do about open source maintainers not getting the funding
they need? It&rsquo;s a very real problem, and one Kevin has <a href="https://twitter.com/kevinverse/status/1032074268291424257">explicitly asked
us</a> to talk about
before we criticise his solution to it. I would be happy to share my thoughts.
I&rsquo;ve struggled for many years to find a way to finance myself as the maintainer
of many dozens of projects. For a long time it has been a demotivating struggle
with no clear solutions, a struggle which at one point probably left me
vulnerable to the temptations offered by the Common Clause. But today, the
situation is clearly improving.</p>
<p>Personally, I have a harder go of it because very little of my open source
software is appealing to the businesses that have the budget to sponsor them.
Instead, I rely on the (much smaller and less stable) recurring donations of my
individual users. When I started accepting these, I did not think that it was
going to work out. But today, I&rsquo;m making far more money from these donations
than I ever thought possible<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>, and I see an upwards trend which will
eventually lead me to being able to work on open source full time. If I were
able to add only a few business-level sponsorships to this equation, I think I
would easily have already reached my goals.</p>
<p>There are other options for securing financing for open source, some of which
Redis has already been exploring. Selling a hosted and supported version of
your service is often a good call. Offering consulting support for your
software has also worked for many groups in the past. Some projects succeed with
(A)GPL for everyone and BSD for a price. These are all better avenues to
explore - making your software proprietary is a tragic alternative that should
not be considered.</p>
<p>We need to combine these methods with a greater appreciation for open source in
the business community. Businesses need engineers - appeal to your peers so they
can appeal to the money on behalf of the projects they depend on. A $250/mo
recurring donation to would be a drop in the bucket of most businesses, but a
major boon to any open source project, with which the business will almost
certainly see tangible value-add as a result. When I get to work today I&rsquo;m going
to identify open source projects we use that accept donations and make the
plea<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>, and keep making the plea week over week until money is spent. You
should, too.</p>
<p>Redis also stands out as a cautionary entry in the history of Contributor
License Agreements. Everyone who has contributed to the now-proprietary Redis
modules has had their hard work stolen and sold by RedisLabs under a proprietary
license. I do not sign CLAs and I think they&rsquo;re a harmful practice for this very
reason. Asking a contributor to sign them is a slap in the face to the good will
which led them to make a contribution in the first place. Don&rsquo;t sign these and
don&rsquo;t ask others to.</p>
<p>I respect antirez<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup> very much, but I am sorely disappointed in him. He should
have known better and, if you&rsquo;re reading this, I urge you to roll back your
misguided decision. But the Commons Clause is much more deeply disturbing. What
Kevin is doing will ruin open source software, maybe for good<sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup>.</p>
<p>I really appreciate some of Kevin&rsquo;s work. <a href="https://fossa.io/">FOSSA</a> is a really
cool tool that can stand to provide some serious value to the open source
community. <a href="https://tldrlegal.com/">TL;DR Legal</a> is a fantastic tool which has
already delivered a tremendous amount of value to open source, and I&rsquo;ve
personally referenced it dozens of times. Thank you, honestly, for your work on
improving the legal landscape of open source. With Commons Clause, however,
Kevin has taken it too far. <a href="https://www.gnu.org/philosophy/free-sw.en.html">The four
freedoms</a> are <em>important</em>. The
only solution is to bury the Common Clause project. Kill the website and GitHub
repository, and we can try to forget this ever happened.</p>
<p>I understand that turning back is going to be hard, which scares me. I know that
Kevin has already put a lot of effort into it and convinced himself that it&rsquo;s
the Right Thing To Do. It takes work to write the clause, vet it for legal
issues, design a website (a beautiful one, I&rsquo;ll give you that), and to promote
it among your target audience. I know how hard it is to distance yourself from
something you&rsquo;ve staked your personal reputation on. You had only the best
intentions<sup id="fnref:6"><a href="#fn:6" class="footnote-ref" role="doc-noteref">6</a></sup>, Kevin, but please step back from the ego and do the right thing -
take this down. You stand to undo all of your hard work for the open source
community in one fell swoop with this initiative. I&rsquo;m begging you, stop while
it&rsquo;s not too late.</p>
<p>Man, two angry articles in a row. I have more technical articles coming up, I
promise.</p>
<hr>
<p><strong>Update 2018-08-23 03:00 UTC:</strong> Richard Stallman of the Free Software
Foundation reached out asking me to clarify the use of &ldquo;open source&rdquo; in this
article. I have refered to the FSF&rsquo;s document on essential freedoms as a
definition of &ldquo;open source&rdquo;. In fact, it is the definition of free software - a
distinct concept. The FSF does not advocate for open source software, but
particularly for free (or &ldquo;libre&rdquo;) software, of which there is some intersection
with open source software. For more information on the difference, refer to
<a href="https://www.gnu.org/philosophy/open-source-misses-the-point.html">Richard&rsquo;s article on the
subject</a>.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Under both the OSI and FSF definitions. The Commons Clause removes freedom 0 of the <a href="https://www.gnu.org/philosophy/free-sw.en.html">four essential freedoms</a>. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p><a href="https://drewdevault.com/donate">Figures here</a> <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>I intend to do an audit, but I have always (and I encourage you to always) kept an eye on the stuff we use as I come across it, looking for opportunities to donate. <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>The maintainer of Redis <a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:5" role="doc-endnote">
<p>As software gets abandoned, making the license more permissive is the last thing on the maintainer&rsquo;s minds. So as the body of Commons Clause software grows, the graveyard will only ever fill. <a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:6" role="doc-endnote">
<p>Honestly, this is a real problem that open source suffers from and I really appreciate the attempt to fix it, misguided as it may have been. But this is not okay, and Kevin needs to recognize the gravity of his mistake and move to correct it. <a href="#fnref:6" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/08/26/Self-hosted-livestreaming.html
title: How to make a self-hosted video livestream
url: https://drewdevault.com/2018/08/26/Self-hosted-livestreaming.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I have seen some articles in the past which explain how to build the ecosystem
<em>around</em> your video streaming, such as live chat and forums, but which leave the
actual video streaming to Twitch.tv. I made a note the last time I saw one of
these articles to write one of my own explaining the video bit. As is often the
case with video, we&rsquo;ll be using the excellent <a href="http://ffmpeg.org/">ffmpeg</a> tool
for this. If it&rsquo;s A/V-related, ffmpeg can probably do it.</p>
<p><em>Note: a demonstration video was previously shown here, but as traffic on this
article died down I took it offline to reduce unnecessary load.</em></p>
<p>ffmpeg has a built-in <a href="https://dashif.org/">DASH</a> output format, which is the
current industry standard for live streaming video to web browsers. It works by
splitting the output up into discrete files and using an <a href="https://drewdevault.com/dash/live.mpd">XML
file</a> (an MPD playlist) to tell the player where they are. Few
browsers support DASH natively, but
<a href="https://github.com/Dash-Industry-Forum/dash.js/wiki">dash.js</a> can polyfill it
by periodically downloading the latest manifest and driving the video element
itself.</p>
<p>Getting the source video into ffmpeg is a little bit beyond the scope of this
article, but I know some readers won&rsquo;t be familiar with ffmpeg so I&rsquo;ll have
mercy. Let&rsquo;s say you want to play some static video files like I&rsquo;m doing above:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">ffmpeg <span class="se">\
</span><span class="se"></span>	-re <span class="se">\
</span><span class="se"></span>	-stream_loop -1 <span class="se">\
</span><span class="se"></span>	-i my-video.mkv <span class="se">\
</span></code></pre></div><p>This will tell ffmpeg to read the input (-i) in real time (-re), and loop it
indefinitely. If instead you want to, for example, use x11grab instead to
capture your screen and pulse to capture desktop audio, try this:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">    -f x11grab <span class="se">\
</span><span class="se"></span>    -r <span class="m">30</span> <span class="se">\
</span><span class="se"></span>    -video_size 1920x1080 <span class="se">\
</span><span class="se"></span>    -i <span class="nv">$DISPLAY</span> <span class="se">\
</span><span class="se"></span>    -f pulse <span class="se">\
</span><span class="se"></span>    -i alsa_input.usb-Blue_Microphones_Yeti_Stereo_Microphone_REV8-00.analog-stereo
</code></pre></div><p>This sets the framerate to 30 FPS and the video resolution to 1080p, then reads
from the X11 display <code>$DISPLAY</code> (usually :0). Then we add pulseaudio and use my
microphone source name, which I obtained with <code>pactl list sources</code>.</p>
<p>Let&rsquo;s add some arguments describing the output format. Your typical web browser
is a finicky bitch and has some very specific demands from your output format if
you want maximum compatability:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">    -codec:v libx264 <span class="se">\
</span><span class="se"></span>    -profile:v baseline <span class="se">\
</span><span class="se"></span>    -level <span class="m">4</span> <span class="se">\
</span><span class="se"></span>    -pix_fmt yuv420p <span class="se">\
</span><span class="se"></span>    -preset veryfast <span class="se">\
</span><span class="se"></span>    -codec:a aac <span class="se">\
</span></code></pre></div><p>This specifices the libx264 video encoder with the baseline level 4 profile, the
most broadly compatible x264 profile, with the yuv420p pixel format, the most
broadly compatible pixel format, the veryfast preset to make sure we can encode
it in realtime, the aac audio codec. Now that we&rsquo;ve specified the parameters for
the output, let&rsquo;s configure the output format: DASH.</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">	-f dash <span class="se">\
</span><span class="se"></span>	-window_size <span class="m">5</span> <span class="se">\
</span><span class="se"></span>	-remove_at_exit <span class="m">1</span> <span class="se">\
</span><span class="se"></span>	/tmp/dash/live.mpd
</code></pre></div><p>The window_size specifies the maximum number of A/V segments to keep in the
manifest at any time, and remove_at_exit will clean up all of the files when
ffmpeg exits. The output file is the path to the playlist to write to disk, and
the segments will be written next to it. The last step is to serve this with
nginx:</p>
<div class="highlight"><pre class="chroma"><code class="language-nginx" data-lang="nginx"><span class="k">location</span> <span class="s">/dash</span> <span class="p">{</span>
        <span class="kn">types</span> <span class="p">{</span>
                <span class="kn">application/dash+xml</span> <span class="s">mpd</span><span class="p">;</span>
                <span class="kn">video/mp4</span> <span class="s">m4v</span><span class="p">;</span>
                <span class="kn">audio/mp4</span> <span class="s">m4a</span><span class="p">;</span>
        <span class="p">}</span>
        <span class="kn">add_header</span> <span class="s">Access-Control-Allow-Origin</span> <span class="s">*</span><span class="p">;</span>
        <span class="kn">root</span> <span class="s">/tmp</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div><p>You can now point the <a href="http://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html">DASH reference
player</a>
at <code>http://your-server.org/dash/live.mpd</code> and see your video streaming there.
Neato! You can add dash.js to your website and you know have a fully self-hosted
video live streaming setup ready to rock.</p>
<p>Perhaps the ffmpeg swiss army knife isn&rsquo;t your cup of tea. If you want to, for
example, use <a href="https://obsproject.com/">OBS Studio</a>, you might want to take a
somewhat different approach. The
<a href="https://github.com/arut/nginx-rtmp-module">nginx-rtmp-module</a> provides an RTMP
(real-time media protocol) server that integrates with nginx. After adding
the DASH output, you&rsquo;ll end up with something like this:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">rtmp <span class="o">{</span>
    server <span class="o">{</span>
        listen 1935<span class="p">;</span>

        application live <span class="o">{</span>
            dash on<span class="p">;</span>
            dash_path /tmp/dash<span class="p">;</span>
            dash_fragment 15s<span class="p">;</span>
        <span class="o">}</span>
    <span class="o">}</span>
<span class="o">}</span>
</code></pre></div><p>Then you can stream to <code>rtmp://your-server.org/live</code> and your dash segments
will show up in <code>/tmp/dash</code>. There&rsquo;s no password protection here, so put it in
the stream URL (e.g. <code>application R9AyTRfguLK8</code>) or use an IP whitelist:</p>
<pre><code>application live {
    allow publish your-ip;
    deny publish all;
}
</code></pre><p>If you want to get creative with it you can use
<a href="https://github.com/arut/nginx-rtmp-module/wiki/Directives#on_publish"><code>on_publish</code></a>
to hit an web service with some details and return a non-2xx code to forbid
streaming. Have fun!</p>
<p>I learned all of this stuff by making a bot which livestreamed Google hangouts
over the LAN to get around the participant limit at work. I&rsquo;ll do a full writeup
about that one later!</p>
<hr>
<p>Here&rsquo;s the full script I&rsquo;m using to generate the live stream on this
page:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh"><span class="cp">#!/bin/sh
</span><span class="cp"></span>rm -f /tmp/playlist
mkdir -p /tmp/dash
<span class="k">for</span> file in /var/www/mirror.sr.ht/hacksway-2018/*
<span class="k">do</span>
	<span class="nb">echo</span> <span class="s2">&#34;file &#39;</span><span class="nv">$file</span><span class="s2">&#39;&#34;</span> &gt;&gt; /tmp/playlist
<span class="k">done</span>

ffmpeg <span class="se">\
</span><span class="se"></span>	-re <span class="se">\
</span><span class="se"></span>	-loglevel error <span class="se">\
</span><span class="se"></span>	-stream_loop -1 <span class="se">\
</span><span class="se"></span>	-f concat <span class="se">\
</span><span class="se"></span>	-safe <span class="m">0</span> <span class="se">\
</span><span class="se"></span>	-i /tmp/playlist <span class="se">\
</span><span class="se"></span>	-vf <span class="s2">&#34;drawtext=\
</span><span class="s2">			fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-Bold.ttf:\
</span><span class="s2">			text=&#39;%{gmtime\:%Y-%m-%d %T} UTC&#39;:\
</span><span class="s2">			fontcolor=white:\
</span><span class="s2">			x=(w-text_w)/2:y=128:\
</span><span class="s2">			box=1:boxcolor=black:\
</span><span class="s2">			fontsize=72,
</span><span class="s2">		drawtext=\
</span><span class="s2">			fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-Bold.ttf:\
</span><span class="s2">			text=&#39;REBROADCAST&#39;:\
</span><span class="s2">			fontcolor=white:\
</span><span class="s2">			x=(w-text_w)/2:y=16:\
</span><span class="s2">			box=1:boxcolor=black:\
</span><span class="s2">			fontsize=48&#34;</span> <span class="se">\
</span><span class="se"></span>	-codec:v libx264 <span class="se">\
</span><span class="se"></span>	-profile:v baseline <span class="se">\
</span><span class="se"></span>	-pix_fmt yuv420p <span class="se">\
</span><span class="se"></span>	-level <span class="m">4</span> <span class="se">\
</span><span class="se"></span>	-preset veryfast <span class="se">\
</span><span class="se"></span>	-codec:a aac <span class="se">\
</span><span class="se"></span>	-f dash <span class="se">\
</span><span class="se"></span>	-window_size <span class="m">5</span> <span class="se">\
</span><span class="se"></span>	-remove_at_exit <span class="m">1</span> <span class="se">\
</span><span class="se"></span>	/tmp/dash/live.mpd
</code></pre></div>


guid: https://drewdevault.com/2018/09/04/Conservative-web-development.html
title: Conservative web development
url: https://drewdevault.com/2018/09/04/Conservative-web-development.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Today I turned off my ad blocker, enabled JavaScript, opened my network monitor,
and clicked the first link on Hacker News - a New York Times article. It started
by downloading a megabyte of data as it rendered the page over the course of
eight full seconds. The page opens with an advertisement 281 pixels tall, placed
before even the title of the article. As I scrolled down, more and more requests
were made, downloading a total of 2.8 MB of data with 748 HTTP requests. An
article was weaved between a grand total of 1419 vertical pixels of ad space,
greater than the vertical resolution of my display. Another 153-pixel ad is
shown at the bottom, after the article. Four of the ads were identical.</p>
<p>I was reminded to subscribe three times, for $1/week (after one year this would
become $3.75/week). One of these reminders attached itself to the bottom of my
screen and followed along as a scrolled. If I scrolled up, it replaced this with
a larger banner, which showed me three other articles and an ad. I was asked for
my email address once, though I would have had to fill out a captcha to submit
it. I took out my phone and repeated the experiment. It took 15 seconds to load,
and I estimate the ads took up a vertical space equal to 4 times my phone&rsquo;s
vertical resolution, each ad alone taking up half of my screen.</p>
<p>The text of the article is a total of 9037 bytes, including the title, author,
and date. I downloaded the images relevant to the article, including the
1477x1082<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> title image. Before I ran them through an optimizer, they weighed
260 KB; after, 236 KB (using only lossless optimizations). 8% of the total
download was dedicated to the content. 5 discrete external companies were
informed of my visit to the page and given the opportunity to run artibrary
JavaScript on it.</p>
<p>If these are the symptoms, what is the cure? My basic principles are these:</p>
<ul>
<li>Use no, or very little, JavaScript</li>
<li>Use raster images sparingly, if at all, and optimize them</li>
<li>Provide interactivity with forms and clever CSS</li>
<li>Identify wasted bandwidth and CPU cycles and optimize them</li>
</ul>
<p>I&rsquo;ve been building <a href="https://meta.sr.ht">sr.ht</a> with these principles in mind,
and I spent a few hours this optimizing it further. What do the results look
like? The heaviest page, <a href="https://meta.sr.ht">the marketing page</a>, today weighs
<strong class="text-info">110 KB</strong> with a cold cache, and <strong
class="text-danger">4.6 KB</strong> warm. <a href="https://github.com/">A similar page</a>
on GitHub.com<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> weighs <strong class="text-info">2900 KB</strong> cold,
<strong class="text-danger">19.4 KB</strong> warm. <a href="https://git.sr.ht/~sircmpwn/linux/tree/master/init/main.c">A more typical
page</a> on sr.ht weighs <strong class="text-info">56.8 KB</strong>
cold and <strong class="text-danger">31.9 KB</strong> warm, after <strong
class="text-warning">2</strong> HTTP requests; on GitHub <a href="https://github.com/torvalds/linux/blob/master/init/main.c">the same
page</a> is <strong class="text-info">781 KB</strong> cold and
<strong class="text-danger">57.4 KB</strong> warm, <strong
class="text-warning">118</strong> requests. This file is 29.1 KB.  The sr.ht
overhead is <strong class="text-info">27.6 KB</strong> cold and <strong
class="text-danger">2.7 KB</strong> warm. The GitHub overhead is respectively
<strong class="text-info">751.9 KB</strong> and <strong class="text-danger">28.2
KB</strong>. There&rsquo;s also a 174-pixel-tall ad on GitHub encouraging me to sign
up for an account, shown before any of the content.</p>
<p>To be fair, the GitHub page has more features. As far as I can tell, most of
these aren&rsquo;t implemented <em>in</em> the page, though, and are rather links to other
pages. Some of the features <em>in</em> the page include a dropdown for filtering
branches and tags, popups that show detail when you hover over an avatars, some
basic interactivity in the search, all things that I can&rsquo;t imagine taking up
much space. Does this justify an order of magnitude increase in resource usage?</p>
<p>Honestly, GitHub does a pretty good job overall. Compared to our New York Times
example, they&rsquo;re downright <em>great</em>. But they could be doing better, and so could
we all. You can build beautiful, interactive websites with HTML and CSS alone,
supported by a simple backend. Pushing the complexity of rendering your
single-page app into the frontend might save you miniscule amounts of
server-side performance, but you&rsquo;d just be offloading the costs onto your
visitor&rsquo;s phone and sucking their battery dry.</p>
<p>There are easy changes you can make. Enable caching on your web server, with a
generous expiry. Use a hash of your resources in the URL so that you can bust
the cache when you need to. Enable gzip for text resources, and HTTP/2. Run your
images through an optimizer, odds are they can be losslessly compressed.  There
are harder changes, too. Design your website to be usable without JavaScript,
and use small amounts of it to enhance the experience - rather than to <em>be</em> the
experience. Use CSS cleverly to provide interactivity<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>. Find ways to offload
work to the server where you can<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>. Measure your pages to look for places to
improve. Challenge yourself to find the simplest way of building the features
you want.</p>
<p>And if anyone at Google is reading, you should try recommending these strategies
for speeding up pages instead of pushing self-serving faux standards like AMP.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Greater than the vertical resolution of my desktop display. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>You may have to log out to see this. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>For example, check out how I implemented the collapsable message details on the <a href="https://lists.sr.ht/~sircmpwn/sr.ht-dev/%3C20180830183221.32377-1-hilobakho%40gmail.com%3E">lists.sr.ht archives</a> <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>I did this when I upgraded to Font Awesome 5 recently. They want you to include some JavaScript to make their SVG icons work, but instead I wrote a <a href="https://git.sr.ht/~sircmpwn/core.sr.ht/tree/70e75e96dc664a1b487ef02cb9936cb8f69105c0/srht/flask.py#L49">dozen lines of Python</a> on the backend which gave me a macro to dump the desired SVG directly into the page. <a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/09/10/Getting-started-with-qemu.html
title: Getting started with qemu
url: https://drewdevault.com/2018/09/10/Getting-started-with-qemu.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I often get asked questions about using my software, particularly sway, on
hypervisors like VirtualBox and VMWare, as well as for general advice on
which hypervisor to choose. My answer is always the same: qemu. There&rsquo;s no
excuse to use anything other than qemu, in my books. But I can admit that it
might be a bit obtuse to understand at first. qemu&rsquo;s greatest strength is also
its greatest weakness: it has so many options that it&rsquo;s hard to know which ones
you need just to get started.</p>
<p>qemu is the swiss army knife of virtualisation, much like ffmpeg is the swiss
army knife of multimedia (which comes as no surprise, given that both are written
by Fabrice Bellard). I run a dozen permanent VMs with qemu, as well as all of
the ephemeral VMs used on <a href="https://meta.sr.ht">builds.sr.ht</a>. Why is it better
than all of the other options? Well, in short: qemu is fast, portable, better
supported by guests, and has more features than Hollywood. There&rsquo;s nothing other
hypervisors can do that qemu can&rsquo;t, and there&rsquo;s plenty qemu can that they
cannot.</p>
<p>Studying the full breadth of qemu&rsquo;s featureset is something you can do over
time. For now, let&rsquo;s break down a simple Linux guest installation. We&rsquo;ll start
by downloading some install media (how about <a href="https://alpinelinux.org/">Alpine
Linux</a>, I like Alpine Linux) and preparing a virtual
hard drive.</p>
<pre><code>curl -O https://nl.alpinelinux.org/alpine/v3.8/releases/x86_64/alpine-standard-3.8.0-x86_64.iso
qemu-img create -f qcow2 alpine.qcow2 16G
</code></pre>
<p>This makes a 16G virtual hard disk in a file named alpine.qcow2, the qcow2
format being a format which appears to be 16G to the guest (VM), but only
actually writes to the host any sectors which were written to by the guest in
practice. You can also expose this as a block device on your local system (or a
remote system!) with qemu-nbd if you need to. Now let&rsquo;s boot up a VM using our
install media and virtual hard disk:</p>
<pre><code>qemu-system-x86_64 \
    -enable-kvm \
    -m 2048 \
    -nic user,model=virtio \
    -drive file=alpine.qcow2,media=disk,if=virtio \
    -cdrom alpine-standard-3.8.0-x86_64.iso \
    -sdl
</code></pre>
<p>This is a lot to take in. Let&rsquo;s break it down:</p>
<p><strong>-enable-kvm</strong>: This enables use of the KVM (kernel virtual machine) subsystem
to use hardware accelerated virtualisation on Linux hosts.</p>
<p><strong>-m 2048</strong>: This specifies 2048M (2G) of RAM to provide to the guest.</p>
<p><strong>-nic user,model=virtio</strong>: Adds a virtual <strong>n</strong>etwork <strong>i</strong>nterface
<strong>c</strong>ontroller, using a virtual LAN emulated by qemu. This is the most
straightforward way to get internet in a guest, but there are other options (for
example, you will probably want to use <code>-nic tap</code> if you want the guest to do
networking directly on the host NIC). <code>model=virtio</code> specifies a special
<code>virtio</code> NIC model, which is used by the virtio kernel module in the guest to
provide faster networking.</p>
<p><strong>-drive file=alpine.qcow2,media=disk,if=virtio</strong>: This attaches our virtual
disk to the guest. It&rsquo;ll show up as <code>/dev/vda</code>. We specify <code>if=virtio</code> for the
same reason we did for <code>-nic</code>: it&rsquo;s the fastest interface, but requires special
guest support from the Linux virtio kernel module.</p>
<p><strong>-cdrom alpine-standard-3.8.0-x86_64.iso</strong> connects a virtual CD drive to the
guest and loads our install media into it.</p>
<p><strong>-sdl</strong> finally specifies the graphical configuration. We&rsquo;re using the SDL
backend, which is the simplest usable graphical backend. It attaches a display
to the guest and shows it in an <a href="https://www.libsdl.org/">SDL</a> window on the
host.</p>
<p>When you run this command, the SDL window will appear and Alpine will boot! You
can complete the Alpine installation normally, using <code>setup-alpine</code> to install
it to the attached disk. When you shut down Alpine, run qemu again without
<code>-cdrom</code> to start the VM.</p>
<p>That covers enough to get you off of VirtualBox or whatever other bad hypervisor
you&rsquo;re using. What else is possible with qemu? Here&rsquo;s a short list of common
stuff you can look into:</p>
<ul>
<li>Running pretty much any guest operating system</li>
<li>Software emulation of non-native architectures like ARM, PPC, RISC-V</li>
<li>Using <code>-spice</code> instead of <code>-sdl</code> to enable remote access to the
display/keyboard/mouse</li>
<li>Read-only disk images with guest writes stored in RAM (<code>snapshot=on</code>)</li>
<li>Non-graphical boot with <code>-nographic</code> and <code>console=ttyS0</code> configured in your
kernel command line</li>
<li>Giving a genuine graphics card to your guest with KVM passthrough for high
performance gaming, OpenCL, etc</li>
<li>Using <a href="https://virt-manager.org/">virt-manager</a> or
<a href="https://help.gnome.org/users/gnome-boxes/stable/">Boxes</a> if you want a GUI to
hold your hand</li>
<li>And much more&hellip;</li>
</ul>
<p>There&rsquo;s really no excuse to be using any other hypervisor<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. They&rsquo;re all
dogshit compared to qemu.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Especially VirtualBox. If you use VirtualBox after reading this article you make poor life choices and are an embarrassment to us all. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/09/30/Sway-wlroots-at-XDC-2018.html
title: Sway & wlroots at XDC 2018
url: https://drewdevault.com/2018/09/30/Sway-wlroots-at-XDC-2018.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Just got my first full night of sleep after the return flight from Spain after
attending <a href="https://xdc2018.x.org/">XDC 2018</a>. It was a lot of fun! I attended
along with four other major wlroots contributors. Joining me were <a href="https://github.com/emersion">Simon Ser
(emersion)</a> (a volunteer) and <a href="https://github.com/ascent12">Scott Anderson
(ascent12)</a> of
<a href="https://www.collabora.com/">Collabora</a>, who work on both
<a href="https://github.com/swaywm/wlroots">wlroots</a> and
<a href="https://github.com/swaywm/sway">sway</a>. <a href="https://github.com/ongy">ongy</a> works on
wlroots, <a href="https://github.com/swaywm/hsroots">hsroots</a>, and
<a href="https://github.com/waymonad/waymonad">waymonad</a>, and joined us on behalf
of <a href="https://www.igel.com/">IGEL</a>. Finally, we were joined by <a href="https://github.com/agx">Guido Günther
(agx)</a> of <a href="https://puri.sm/">Purism</a>, who works with us
on wlroots and on the Librem 5. This was my first time meeting most of them
face-to-face!</p>
<p>wlroots was among the highest-level software represented at XDC. Most of the
attendees are hacking on the kernel or mesa drivers, and we had a lot to learn
from each other. The most directly applicable talk was probably VKMS (virtual
kernel mode setting), a work-in-process kernel subsystem which will be useful
for testing the complex wlroots DRM code. We had many chances to catch up with
the presenters after their talk to learn more and establish a good
relationship. We discovered from these chats that some parts of our DRM code
was buggy, and have even started onboarding some of them as contributors to sway
and wlroots.</p>
<p>We also learned a lot from the other talks, in ways that will pay off over time.
One of my favorites was an introduction to the design of Intel GPUs, which went
into a great amount of detail into how the machine code for these GPUs worked,
why these design decisions make them efficient, and their limitations and
inherent challenges. Combined with other talks, we got a lot of insight into the
design and function of mesa, graphics drivers, and GPUs. These folks were very
available to us for further discussion and clarification after their talks, a
recurring theme at XDC and one of the best parts of the conference.</p>
<p>Another recurring theme at XDC was talks about how mesa is tested, with the most
in-depth coverage being on Intel&rsquo;s new CI platform. They provide access to Mesa
developers to test their code on <em>every</em> generation of Intel GPU in the course
of about 30 minutes, and brought some concrete data to the table to show that it
really works to make their drivers more stable. I took notes that you can expect
to turn into builds.sr.ht features! And since these folks were often available
for chats afterwards, I think they were taking notes, too.</p>
<p>I also met many of the driver developers from AMD, Intel, and Nvidia; all of
whom had interesting insights and were a pleasure to hang out with. In fact,
Nvidia&rsquo;s representatives were the first people I met! On the night of the
kick-off party, I led the wlroots clan to the bar for beers and introduced
myself to the people who were standing there - who already knew me from my
writings critical of Nvidia. Awkward! A productive conversation ensued
regardless, where I was sad to conclude that we still aren&rsquo;t going to see any
meaningful involvement in open source from Nvidia. Many of their engineers are
open to it, but I think that the engineering culture at Nvidia is unhealthy and
that the engineers have very little influence. We made our case and brought up
points they weren&rsquo;t thinking about, and I can only hope they&rsquo;ll take them home
and work on gradually improving the culture.</p>
<p>Unfortunately, Wayland itself was somewhat poorly represented. Daniel Stone (a
Wayland &amp; Weston maintainer) was there, and Roman Glig (of KDE), but some KDE
folks had to cancel and many people I had hoped to meet were not present. Some
of the discussions I wanted to have about protocol standardization and
cooperation throughout Wayland didn&rsquo;t happen. Regardless, the outcome of XDC was
very positive - we learned a lot and taught a lot. We found new contributors to
our projects, and have been made into new contributors for everyone else&rsquo;s
projects.</p>
<p>Big shoutout to the X Foundation for organizing the event, and to the beautiful
city of A Coruña for hosting us, and to University of A Coruña for sharing their
university - which consequently led to meeting some students there that used
Sway and wanted to contribute! Thanks as well to the generous sponsors, both for
sponsoring the event and for sending representatives to give talks and meet the
community.</p>


guid: https://drewdevault.com/2018/10/05/Dont-sign-a-CLA.html
title: Don't sign a CLA
url: https://drewdevault.com/2018/10/05/Dont-sign-a-CLA.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>A large minority of open-source projects come with a CLA, or Contributor License
Agreement, and require you to sign one before they&rsquo;ll merge your patch. These
agreements typically ask you to go above and beyond the rights you afford the
project by contributing under the license the software is distributed with. And
you should never sign one.</p>
<p>Free and open source software licenses grant explicit freedoms to three groups:
the maintainers, the users, <em>and</em> the contributors. An important freedom is the
freedom to make changes to the software and to distribute these changes to the
public. The natural place to do so is by contributing to the upstream project,
something a project should be thankful for. A CLA replaces this gratitude with
an attempt to weaken these freedoms in a manner which may stand up to the letter
of the license, but is far from the spirit.</p>
<p>A CLA is a kick in the groin to a contributor&rsquo;s good-faith contribution to the
project. Many people, myself included, contribute to open source projects under
the assumption that my contributions will help serve a project which continues
to be open source in perpetuity, and a CLA provides a means for the project
maintainers to circumvent that. What the CLA is actually used for is to give the
project maintainers the ability to relicense your work under a more restrictive
software license, up to and including making it entirely closed source.</p>
<p>We&rsquo;ve seen this happen before. Consider the Redis Labs debacle, where they
adopted the nonfree<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> Anti-Commons Clause<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>, and used their CLA to pull along
any external contributions for the ride. As thanks for the generous time
invested by their community into their software, they yank it out from
underneath it and repurpose it to make money with an obscenely nonfree product.
Open source is a commitment to your community. Once you make it, you cannot take
it back. You don&rsquo;t get the benefits associated with being an open source project
if you have an exit hatch. You may argue that it&rsquo;s your right to do what you
want with your project, but making it open source is <em>explicitly waiving that
right</em>.</p>
<p>So to you, the contributor: if you are contributing to open source and you want
it to stay that way, you should not sign a CLA. When you send a patch to a
project, you are affording them the same rights they afforded you. The
relationship is one of equals. This is a healthy balance. When you sign a CLA,
you give them unequal power over you. If you&rsquo;re scratching an itch and just
want to submit the patch in good faith, it&rsquo;s easy enough to fork the project and
put up your changes in a separate place. This is a right afforded to you by
every open source license, and it&rsquo;s easy to do. Anyone who wants to use your
work can apply your patches on top of the upstream software. Don&rsquo;t sign away
your rights!</p>
<hr>
<p>Additional reading: <a href="https://blog.hansenpartnership.com/gpl-as-the-best-licence-governance-and-philosophy/">GPL as the Best Licence – Governance and Philosophy</a></p>
<p>Some responses to the discussion around this article:</p>
<p><em>What about the <a href="https://www.apache.org/licenses/cla-corporate.txt">Apache Foundation
CLA</a>?</em> This CLA is one of the
better ones, because it doesn&rsquo;t transfer copyright over your work to the Apache
Foundation. I have no beef with clauses 1 and 3-8. However, term 2 is too broad
and I would not sign this CLA.</p>
<p><em>What about the Linux kernel <a href="https://elinux.org/Developer_Certificate_Of_Origin">developer certificate of
origin</a>?</em> I applaud the
Linux kernel&rsquo;s approach here. It covers their bases while still strongly
protecting the rights of the patch owner. It&rsquo;s a short statement with little
legalese and little fanfare to agreeing to it (just add &ldquo;Signed-off By&rdquo; to your
commit message). I approve.</p>
<hr>
<p>Update April 2021: I wrote a follow-up article about the Developer Certificate
of Origin in particular: <a href="https://drewdevault.com/2021/04/12/DCO.html">The Developer Certificate of Origin is a great alternative to a CLA</a></p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p><a href="https://drewdevault.com/2018/08/22/Commons-clause-will-destroy-open-source.html">Free as in freedom</a> <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Call me petty, but I can&rsquo;t in good faith call it the &ldquo;Commons Clause&rdquo; when its purpose is to <em>remove</em> software from the commons. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/10/08/Go-1.11.html
title: Go 1.11 got me to stop ignoring Go
url: https://drewdevault.com/2018/10/08/Go-1.11.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I took a few looks at Go over the years, starting who knows when. My first
serious attempt to sit down and learn some damn Go was in 2014, when I set a new
personal best at almost 200 lines of code before I got sick of it. I kept
returning to Go because I could see how much potential it had, but every time I
was turned off for the same reason: <code>GOPATH</code>.</p>
<p>You see, <code>GOPATH</code> crossed a line. Go is opinionated, which is fine, but with
<code>GOPATH</code> its opinions extended beyond my Go work and into the rest of my system.
As a naive new Go user, I was prepared to accept their opinions on faith - but
only within their domain. I already have opinions about how to use my computer.
I knew Go was cool, but it could be the second coming of Christ, and so long as
it was annoying to use and didn&rsquo;t integrate with my workflow, I (rightfully)
wouldn&rsquo;t care.</p>
<p>Thankfully Go 1.11 solves this problem, and solves it delightfully well. I can
now keep Go&rsquo;s influence contained to the Go projects I work with, and in that
environment I&rsquo;m much more forgiving of anything it wants to do. And when
considered in the vacuum of Go, what it wants to do is really compelling. Go
modules are <em>great</em>, and probably the single best module system I&rsquo;ve used in any
programming language. Go 1.11 took my biggest complaint and turned it into one
of my biggest compliments. Now that the One Big Problem is gone, I&rsquo;ve really
started to appreciate Go. Let me tell you about it.</p>
<p>The most important feature of Go is its simplicity. The language is small and
it grows a small number of features in each release, which rarely touch the
language itself. Some people see this as stagnation, but I see it as stability
and I know that very little Go code in the wild, no matter how old, is going to
be unidiomatic or fail to compile. Even setting aside stability, the
conservative design of the language makes Go code in the wild remarkably
consistent. Almost all third-party Go libraries are high quality stuff. Gofmt
helps with this as well<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. The limitations of the language and the way the
stdlib gently nudges you into good patterns make it easy to write good Go code.
Most of the &ldquo;bad&rdquo; Go libraries I&rsquo;ve found are trying to work around Go&rsquo;s
limitations instead of embracing them.</p>
<p>There&rsquo;s more. The concurrency model is superb. It should come as no surprise
that a language built by the alumni of Plan 9 would earn high marks in this
regard, and consequently you can scale your Go program up to be as concurrent as
you want without even thinking about it. The standard library is also excellent -
designed consistently and designed well, and I can count on one hand (or even
one finger) the number of stdlib modules I&rsquo;ve encountered that feel crusty. The
type system is great, too. It&rsquo;s the perfect balance of complexity and simplicity
that often effortlessly grants these traits to the abstractions you make with
it.</p>
<p>I&rsquo;m not even slightly bothered by the lack of generics - years as a C programmer
taught me not to need them, and I think most of the cases where they&rsquo;re useful
are to serve designs which are too complicated to use anyway. I do have some
complaints, though. The concurrency model is great, but a bit too magical and
implicit.  Error handling is annoying, especially because finding the origin of
the error is unforgivably difficult, but I don&rsquo;t know how to improve it. The log
module leaves a lot to be desired and can&rsquo;t be changed because of legacy
support.  <code>interface{}</code> is annoying when you have to deal with it, like when
dealing with JSON you can&rsquo;t unmarshall into a struct.</p>
<p>My hope for the future of Go is that it will continue to embrace simplicity in
the face of cries for complexity. I consider Go modules a runaway success
compared to dep, and I hope to see this story repeated<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> before hastily adding
generics, better error handling, etc. Go doesn&rsquo;t need to compete with anyone
like Rust, and trying to will probably ruin what makes Go great. My one request
of the Go team: don&rsquo;t make changes in Go 2.0 which make the APIs of existing
libraries unidiomatic.</p>
<p>Though I am growing very fond of it, by no means am I turning into a Go zealot.
I still use C, Python, and more all the time and have no intention of stopping.
A programming language which tries to fill all niches is a failed programming
language. But, to those who were once like me: Go is good now! In fact, it&rsquo;s
great! Try it!</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>I have <em>minor</em> gripes with gofmt, but the benefits make up for it beautifully. On the other hand, I have <em>major</em> gripes with PEP-8, and if you ever see me using it I want you to shoot me in the face. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Though hopefully with less drama. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/10/20/Sway-1.0-highlights.html
title: Sway 1.0-beta.1 release highlights
url: https://drewdevault.com/2018/10/20/Sway-1.0-highlights.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>1,173 days ago, I wrote sway&rsquo;s <a href="https://github.com/swaywm/sway/commit/6a33e1e3cddac31b762e4376e29c03ccf8f92107">initial commit</a>, and 8,269 commits
followed<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>, written by hundreds of contributors. What started as a side
project became the most fully featured and stable Wayland desktop available, and
drove the development of what has become the dominant solution for building
Wayland compositors - <a href="https://github.com/swaywm/wlroots">wlroots</a>, now the
basis of 10 Wayland compositors.</p>
<p>Sway 1.0-beta.1 was just released and is 100% compatible with the <a href="https://i3wm.org/">i3 X11 window
manager</a>. It&rsquo;s faster, prettier, sips your battery, and
supports <a href="https://wayland.freedesktop.org/">Wayland</a> clients. When we started, I
honestly didn&rsquo;t think we&rsquo;d get here. When I decided we&rsquo;d rewrite our internals
and build wlroots over a year ago, I didn&rsquo;t think we&rsquo;d get here. It&rsquo;s only
thanks to an amazing team of talented contributors that we did. So what can
users expect from this release? The difference between sway 0.15 and sway 1.0 is
like night and day. The annoying bugs which plauged sway 0.15 are gone, and in
their place is a rock solid Wayland compositor with loads of features you&rsquo;ve
been asking after for years. The <a href="https://github.com/swaywm/sway/releases/tag/1.0-beta.1">official release
notes</a> are a bit thick,
so let me give you a guided tour.</p>
<h2 id="new-output-features">New output features</h2>
<p>Outputs, or displays, grew a lot of cool features in sway 1.0. As a reminder,
you can get the names of your outputs for use in your config file by using
<code>swaymsg -t get_outputs</code>. What can you do with them?</p>
<p>To rotate your display 90 degrees, use:</p>
<pre><code>output DP-1 transform 90
</code></pre>
<p>To enable our improved HiDPI support<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>, use:</p>
<pre><code>output DP-1 scale 2
</code></pre>
<p>Or to enable fractional scaling (see man page for warnings about this):</p>
<pre><code>output DP-1 scale 1.5
</code></pre>
<p>You can also now run sway on multiple GPUs. It will pick a primary GPU
automatically, but you can override this by specifying a list of card names at
startup with <code>WLR_DRM_DEVICES=card0:card1:...</code>. The first one will do all of the
rendering and any displays connected to subsequent cards will have their buffers
copied over.</p>
<p>Other cool features include support for daisy-chained DisplayPort configurations
and improved Redshift support. Also, the long annoying single-output limitation
of wlc is behind us: you can now drag windows between outputs with the mouse.</p>
<p>See <code>man 5 sway-output</code> for more details on configuring these features.</p>
<h2 id="new-input-features">New input features</h2>
<p>Input devices have also matured a lot. You can get a list of their identifiers
with <code>swaymsg -t get_inputs</code>. One oft requested feature was a better way of
configuring your keyboard layout, which you can now do in your config file:</p>
<pre><code>input &quot;9456:320:Metadot_-_Das_Keyboard_Das_Keyboard&quot; {
    xkb_options caps:escape
    xkb_numlock enabled
}
</code></pre><p>We also now support drawing tablets, which you can bind to a specific output:</p>
<pre><code>input &quot;1386:827:Wacom_Intuos_S_2_Pen&quot; {
    map_to_output DP-3
}
</code></pre><p>You can also now do crazy stuff like having multiple mice with multiple cursors,
and linking keyboards, mice, drawing tablets, and touchscreens to each other
arbitrarily. You can now have your dvorak keyboard for normal use and a second
qwerty keyboard for when your coworker comes over for a pair programming
session. You can even give your coworker the ability to focus and type into
<em>separate</em> windows from what you&rsquo;re working on.</p>
<h2 id="third-party-panels-lockscreens-and-more">Third-party panels, lockscreens, and more</h2>
<p>Our new layer-shell protcol is starting to take hold in the community, and
enables the use of even more third-party software on sway. One of our main
commitments to you for sway 1.0 and wlroots is to break the boundaries between
Wayland compositors and encourange standard interopable protocols - and we&rsquo;ve
done so. Here are some interesting third-party layer-shell clients in the wild:</p>
<ul>
<li><a href="https://github.com/Alexays/Waybar">Waybar</a>, a new panel</li>
<li><a href="https://github.com/emersion/mako">mako</a>, a notification daemon</li>
<li><a href="https://source.puri.sm/Librem5/virtboard">virtboard</a>, an on-screen keyboard</li>
<li><a href="https://github.com/emersion/slurp">slurp</a>, a tool to interactively select a
region of the screen</li>
<li><a href="https://source.puri.sm/Librem5/phosh">Phosh</a>, the <a href="https://puri.sm/">Purism</a>
team&rsquo;s shell for their <a href="https://puri.sm/shop/librem-5/">Librem 5</a> phone</li>
</ul>
<p>We also added two new protocols for capturing your screen: screencopy and
dmabuf-export, respectively these are useful for screenshots and real-time
screen capture, for example to live stream on Twitch. Some third-party software
exists for these, too:</p>
<ul>
<li><a href="https://github.com/emersion/grim">grim</a>, for taking screenshots</li>
<li><a href="https://github.com/atomnuker/wlstream">wlstream</a>, for recording video</li>
</ul>
<h2 id="dpms-auto-locking-and-idle-management">DPMS, auto-locking, and idle management</h2>
<p>Our new <code>swayidle</code> tool adds support for all of these features, and even works
on other Wayland compositors. To configure it, start by running the daemon in
your sway config file:</p>
<pre><code>exec swayidle \
    timeout 300 'swaylock -c 000000' \
    timeout 600 'swaymsg &quot;output * dpms off&quot;' \
       resume 'swaymsg &quot;output * dpms on&quot;' \
    before-sleep 'swaylock -c 000000'
</code></pre><p>This example will, after 300 seconds of inactivity, lock your screen. Then after
600 seconds, it will turn off all of your outputs (and turn them back on when
you wiggle the mouse). This configuration also locks your screen before your
system goes to sleep. None of this will happen if you&rsquo;re watching a video on a
supported media player (mpv, for example). For more details check out <code>man swayidle</code>.</p>
<h2 id="miscellaneous-bits">Miscellaneous bits</h2>
<p>There are a few other cool features I think are worth briefly mentioning:</p>
<ul>
<li><code>bindsym --locked</code></li>
<li>swaylock has a config file now</li>
<li>Drag and drop is supported</li>
<li>Rich content (like images) is synced between the Wayland and X11 clipboards</li>
<li>The layout is updated atomically, meaning that you&rsquo;ll never see an in-progress
frame when resizing windows</li>
<li>Primary selection is implemented and synced with X11</li>
<li>Pretty much every long-standing bug has been fixed</li>
</ul>
<p>For the full run-down see the <a href="https://github.com/swaywm/sway/releases/tag/1.0-beta.1">release
notes</a>. Give the beta a
try, and we&rsquo;re all looking forward to sway 1.0!</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>5,044 sway commits and 3,225 wlroots commits at the time of writing. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Sway now has the best HiDPI support on Linux, period. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/10/29/How-does-virtual-memory-work.html
title: How does virtual memory work?
url: https://drewdevault.com/2018/10/29/How-does-virtual-memory-work.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Virtual memory is an essential part of your computer, and has been for several
decades. In my <a href="https://drewdevault.com/2016/05/28/Understanding-pointers.html">earlier article on pointers</a>, I compared memory to a
giant array of octets (bytes), and explained some of the abstractions we make
on top of that. In actual fact, memory is more complicated than a flat array of
bytes, and in this article I&rsquo;ll explain how.</p>
<p>An astute reader of my earlier article may have considered that pointers on,
say, an x86_64 system, are 64 bits long<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. With this, we can address up to
18,446,744,073,709,551,616 bytes (16 exbibytes<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>) of memory. I only have 16
GiB of RAM on this computer, so what gives? What&rsquo;s the rest of the address space
for? The answer: all kinds of things! Only a small subset of your address space
is mapped to physical RAM. A system on your computer called the MMU, or Memory
Management Unit, is responsible for managing the abstraction that enables this
and other uses of your address space. This abstraction is called virtual memory.</p>
<p>The kernel interacts directly with the MMU, and provides syscalls like
[mmap(2)][mmap] for userspace programs to do the same. Virtual memory is
typically allocated a page at a time, and given a purpose on allocation, along
with various flags (documented on the mmap page). When you call <code>malloc</code>, libc
uses the mmap syscall to allocate pages of heap, then assigns a subset of that
to the memory you asked for. However, since many programs can run concurrently
on your system and may request pages of RAM at any time, your physical RAM can
get fragmented.  Each time the kernel hits a context switch<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>, it swaps out
the page table for the next process.</p>
<p>This is used in this way to give each process its own clean address space and to
provide memory isolation between processes, preventing them from accessing each
other&rsquo;s memory. Sometimes, however, in the case of shared memory, the same
physical memory is deliberately shared with multiple processes.  Many pages can
also be any combination readable, writable, or executable - the latter meaning
that you could jump to it and execute it as native code.  Your compiled program
is a file, after all - mmap some executable pages, load it into memory, jump to
it, and huzzah: you&rsquo;re running your program<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>. This is how JITs, dynamic
recompiling emulators, etc, do their job. A common way to reduce risk here,
popular on *BSD, is enforcing W^X (writable XOR executable), so that a page can
be either writable or executable, but never both.</p>
<p>Sometimes all of the memory you think you have isn&rsquo;t actually there, too. If you
blow your RAM budget across your whole system, swap gets involved. This is when
pages of RAM are &ldquo;swapped&rdquo; to disk - as soon as your program tries to access it
again, a page fault occurs, transferring control to the kernel. The kernel
restores from swap, damning some other poor process to the fate, and returns
control to your program.</p>
<p>Another very common use for virtual memory is for memory mapped I/O. This can
be, for example, mapping a file to memory so you can efficiently read and write
to disk. You can map other sorts of hardware, too, such as video memory. On 8086
(which is what your computer probably pretends to be when it initially
boots<sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup>), a simple 96x64 cell text buffer is available at address <code>0xB8000</code>.
On my TI-Nspire CX calculator, I can read the current time from the real-time
clock at <code>0x90090000</code>.</p>
<p>In summary, MMUs arrived almost immediately on the computing scene, and have
become increasingly sophisticated ever since. Virtual memory is a powerful tool
which grants userspace programmers elegant, convenient, and efficient access to
the underlying hardware.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Fun fact: most x86_64 implementations actually use 48 bit addresses internally, for a maximum theoretical limit of 256 TiB of RAM. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>I had to look that SI prefix up. This number is 2<sup>64</sup>, by the way. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>This means to switch between which process/thread is currently running on a single CPU. I&rsquo;ll write an article about this sometime.
[mmap]: <a href="http://man7.org/linux/man-pages/man2/mmap.2.html">http://man7.org/linux/man-pages/man2/mmap.2.html</a> <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>There are actually at least a dozen other steps involved in this process. I&rsquo;ll write an article about loaders at some point, too. <a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:5" role="doc-endnote">
<p>You can make it stop pretending to do this with <a href="https://wiki.osdev.org/Protected_Mode">an annoying complicated sequence of esoteric machine code instructions</a>. An even more annoying sequence is required to <a href="https://wiki.osdev.org/Setting_Up_Long_Mode">enter 64-bit mode</a>. It gets even better if you want to set up <a href="https://wiki.osdev.org/Symmetric_Multiprocessing">multiple CPU cores</a>! <a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/10/30/Its-not-okay-to-pretend-youre-open-source.html
title: It's not okay to pretend your software is open source
url: https://drewdevault.com/2018/10/30/Its-not-okay-to-pretend-youre-open-source.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Unfortunately, I find myself writing about the Commons Clause again. For those
not in the know, the Commons Clause is an addendum designed to be added to free
software licenses. The restrictions it imposes (you cannot sell the software)
makes the resulting franken-license nonfree. I&rsquo;m not going to link to the
project which brought this subject back into the discussion - they don&rsquo;t deserve
the referral - but the continued proliferation of software using the Commons
Clause gives me reason to speak out against it some more.</p>
<p>One of my largest complaints with the Commons Clause is that it hijacks
language used by open source projects to proliferate nonfree software, and
encourages software using it to do the same. Instead of being a new software
license, it tries to stick itself onto other respected licences - often the
Apache 2.0 license. The name, &ldquo;Commons Clause&rdquo;, is also disingenuous, hijacking
language used by respected entities like Creative Commons. In truth, the Commons
Clause serves to remove software from the commons<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. Combining these
problems gives you language like &ldquo;Apache+Commons Clause&rdquo;, which is easily
confused with [Apache Commons][apache-commons].</p>
<p>Projects using the Commons Clause have also been known to describe their license
as &ldquo;permissive&rdquo; or &ldquo;open&rdquo;, some even calling their software &ldquo;open source&rdquo;. This
is dishonest. FOSS refers to &ldquo;free and open source software&rdquo;. The former, free
software, is defined by the <a href="https://www.gnu.org/philosophy/free-sw.en.html">free software definition</a>, published by
<a href="https://gnu.org">GNU</a>. The latter, open source software, is defined by the <a href="https://opensource.org/osd">open source
definition</a>, published by the <a href="https://opensource.org">OSI</a>. Their definitions are very
similar, and nearly all FOSS licenses qualify under both definitions. These are
unambiguous, basic criteria which protects developers, contributors, and users
of free and open source software. These definitions are so basic, important and
well-respected that dismissing them is akin to dismissing climate change.</p>
<p>Claiming your software is open source, permissively licensed, free software,
etc, when you use the Commons Clause, is <em>lying</em>. These lies are pervasive among
users of the Commons Clause. The page listing <a href="https://redis.io/modules">Redis
Modules</a>, for example, states that only software under
an OSI-approved license is listed. Six of the modules there are using nonfree
licenses, and antirez seems content to <a href="https://github.com/antirez/redis-doc/pull/984">ignore the problem</a> until <a href="https://github.com/RedisLabsModules/RediSearch/issues/518">we
forget about it</a>. They&rsquo;re in for a long wait - we&rsquo;re not going to
forget about <strong>shady, dishonest, and unethical companies like Redis Labs</strong>.</p>
<p>I don&rsquo;t use nonfree software<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>, but I&rsquo;m not going to sit here and tell you
not to make nonfree software. You have every right to license your work in any
way you choose. However, if you choose not to use a FOSS license, you need to
own up to it. Don&rsquo;t pretend that your software is something it&rsquo;s not. There are
many benefits to being a member of the free software community, but you are not
entitled to them if your software isn&rsquo;t. This behavior has to stop.</p>
<p>Finally, I have some praise to offer. <a href="https://dgraph.io/">Dgraph</a> was briefly
licensed under Apache plus the Commons Clause, and had the sort of misleading
and false information this article decries on their marketing website, docs, and
so on. However, they&rsquo;ve rolled it back, and Dgraph is now using the Apache 2.0
license with no modifications.  Thank you!</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>This is why I often refer to it as the &ldquo;Anti-Commons Clause&rdquo;, though I felt that was a bit too Stallman-esque for this article.
[apache-commons]: <a href="http://commons.apache.org/">http://commons.apache.org/</a> <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Free as in freedom, not as in free beer. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/11/15/sr.ht-general-availability.html
title: sr.ht, the hacker's forge, now open for public alpha
url: https://drewdevault.com/2018/11/15/sr.ht-general-availability.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I&rsquo;m happy to announce today that I&rsquo;m opening <a href="https://sr.ht">sr.ht</a> (pronounced
&ldquo;sir hat&rdquo;, or any other way you want) to the general public for the remainder of
the alpha period. Though it&rsquo;s missing some of the features which will be
available when it&rsquo;s completed, sr.ht today represents a very capable software
forge which is already serving the needs of many projects in the free &amp; open
source software community. If you&rsquo;re familiar with the project and ready to
register your account, you can head straight to <a href="https://sr.ht">the sign up
page</a>.</p>
<p>For those who are new, let me explain what makes sr.ht special. It provides many
of the trimmings you&rsquo;re used to from sites like GitHub, Gitlab, BitBucket, and
so on, including git repository hosting, bug tracking software, CI, wikis, and
so on. However, the sr.ht model is different from these projects - where many
forges attempt to replicate GitHub&rsquo;s success with a thinly veiled clone of the
GitHub UI and workflow, sr.ht is fundamentally different in its approach.</p>
<blockquote>
<p>The sr.ht platform excites me more than any project in recent memory. It’s a
fresh concept, not a Github wannabe like Gitlab. I always thought that if
something is going to replace Github it would have to be a paradigm change, and
I think that’s what we’re seeing here. Drew’s project blends the wisdom of the
kernel hackers with a tasteful web interface.</p>
</blockquote>
<div style="margin-top: -1rem; margin-bottom: 1rem"><small>&mdash;<a href="https://lobste.rs/s/h1udkf/git_is_already_federated_decentralized#c_smnkic">begriffs on lobste.rs</a></small></div>
<p>The 500 foot view is that sr.ht is a <a href="https://git.sr.ht/~sircmpwn/?search=sr.ht">100% free and open
source</a> software forge, with a hosted
version of the services running <em>at</em> <a href="https://sr.ht">sr.ht</a> for your
convenience. Unlike GitHub, which is almost entirely closed source, and Gitlab,
which is mostly open source but with a proprietary premium offering, all of
sr.ht is completely open source, with a copyleft license<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. You&rsquo;re welcome
to install it on your own hardware, and <a href="https://man.sr.ht/installation.md">detailed
instructions</a> are available for those who
want to do so. You can also send patches upstream, which are then integrated
into the hosted version.</p>
<p>sr.ht is special because it&rsquo;s extremely modular and flexible, designed with
interoperability with the rest of the ecosystem in mind. On top of that, sr.ht
is one of the most lightweight websites on the internet, with the average page
weighing less than 10 KiB, with <strong>no tracking</strong> and <strong>no JavaScript</strong>. Each
component - git hosting, continuous integration, etc - is a standalone piece of
software that integrates deeply with the rest of sr.ht <em>and</em> with the rest of
the ecosystem outside of sr.ht. For example, you can use builds.sr.ht to compile
your GitHub pull requests, or you can keep your repos on git.sr.ht and host
everything in one place. Unlike GitHub, which favors its own in-house pull
request workflow<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>, sr.ht embraces and improves upon the email-based
workflow favored by git itself, along with many of the more hacker-oriented
projects around the net. I&rsquo;ve put a lot of work into making this powerful
workflow more <a href="https://man.sr.ht/git.sr.ht/send-email.md">accessible and
comprehensible</a> to the average
hacker.</p>
<p>The flagship product from sr.ht is its continuous integration platform,
builds.sr.ht, which is easily the most capable continuous integration system
available today. It&rsquo;s so powerful that I&rsquo;ve been working with multiple Linux
distributions on bringing them onboard because it&rsquo;s the only platform which can
scale to the automation needs of an entire Linux distribution. It&rsquo;s so powerful
that I&rsquo;ve been working with maintainers of <em>non-Linux</em> operating systems, from
BSD to even Hurd, because it&rsquo;s the only platform which can even consider
supporting their needs. Smaller users are loving it, too, many of whom are
jumping ship from Travis and Jenkins in favor of the simplicity and power of
builds.sr.ht.</p>
<p>On builds.sr.ht, simple YAML-based <a href="https://man.sr.ht/builds.sr.ht/#build-manifests">build
manifests</a>, similar to those
you see on other platforms, are used to describe your builds. You can submit
these through the web, the API, or various integrations.  Within seconds, a
virtual machine is booted with KVM, your build environment is sent to it, and
your scripts start running. A diverse set of base images are supported on a
variety of architectures, soon to include the first hardware-backed RISC-V
cycles available to the general public. builds.sr.ht is used to automate
everything from the deployment of this Jekyll-based blog, testing GitHub pull
requests for <a href="https://swaywm.org">sway</a>, building and testing packages for
<a href="https://postmarketos.org/">postmarketOS</a>, and deploying complex applications
like builds.sr.ht itself. Our base images <a href="https://builds.sr.ht/~sircmpwn/alpine/edge">build, test, and deploy
themselves</a> every day.</p>
<p>The lists.sr.ht service is another important part of sr.ht, and a large part of
how sr.ht embraces the model used by major projects like Linux, Postgresql, git
itself, and many more. lists.sr.ht finally modernizes mailing lists, with a
powerful and elegant web interface for hacking on and talking about your
projects. Take a look at the <a href="https://lists.sr.ht/~sircmpwn/sr.ht-dev">sr.ht-dev</a> list to see patches
developed for sr.ht itself. Another good read is the <a href="https://lists.sr.ht/~emersion/mrsh-dev">mrsh-dev</a> list,
used for development on the <a href="https://git.sr.ht/~emersion/mrsh">mrsh</a> project, or my own <a href="https://lists.sr.ht/~sircmpwn/public-inbox">public
inbox</a>, where I take comments for this blog and grab-bag
discussions for my smaller projects.</p>
<p>I&rsquo;ve just scratched the surface, and there&rsquo;s much more for you to discover. You
could look at my <a href="https://git.sr.ht/~sircmpwn/scdoc">scdoc</a> project to get an
idea of how the git browser looks and feels. You could <a href="https://todo.sr.ht/~sircmpwn">browse tickets on my
todo.sr.ht profile</a> to get a feel for the bug
tracking software. Or you could check out the <a href="https://man.sr.ht">detailed
manual</a> on sr.ht&rsquo;s git-powered wiki service. You could also
just <a href="https://sr.ht">sign up for an account</a>!</p>
<p>sr.ht isn&rsquo;t complete, but it&rsquo;s maturing fast and I think you&rsquo;ll love it. Give it
a try, and I&rsquo;m only <a href="mailto:sir@cmpwn.com">an email away</a> to receive your
feedback.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Some components use the 3-clause BSD license. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>A model that many have replicated in their own GitHub alternatives. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/12/04/How-to-abandon-a-FLOSS-project.html
title: How to abandon a FLOSS project
url: https://drewdevault.com/2018/12/04/How-to-abandon-a-FLOSS-project.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>It&rsquo;s no secret that maintaining free and open source software is often
a burdensome and thankless job. I empathise with maintainers who lost interest
in a project, became demotivated by the endless demands of users, or are no
longer blessed with enough free time. Whatever the reason, FLOSS work is
volunteer work, and you&rsquo;re free to stop volunteering at any time.</p>
<p>In my opinion, there are two good ways to abandon a project: the <em>fork it</em>
option and the <em>hand-off</em> option. The former is faster and easier, and you can
pick this if you want to wash your hands of the project ASAP, but has a larger
effect on the community. The latter is not always possible, requires more work
on your part, and takes longer, but it has a minimal impact on the community.</p>
<p>Let&rsquo;s talk about the easy way first. Start by adding a notice to your README
that your software is now unmaintained. If you have the patience, give a few
weeks notice before you really stop paying attention to it. Inform interested
parties that they should consider forking the software and maintaining it
themselves under another name. Once a fork gains traction, update the README
again to direct would-be users to the fork. If no one forks it, you could
consider directing users to similar alternatives to your software.</p>
<p>This approach allows you to quickly absolve yourself of responsibility. Your
software is no worse than it was yesterday, which allows users a grace period to
collect themselves and start up a fork. If you revisit your work later, you can
also become a contributor to the fork yourself, which removes the stress of
being a maintainer while still providing value to the project. Or, you can just
wash your hands of it entirely and move on to bigger and better things. This
&ldquo;fork it&rdquo; approach is safer than giving control of your project to passerby,
because it requires your users to acknowledge the transfer of power, instead of
being surprised by a new maintainer in a trusted package.</p>
<p>The &ldquo;fork it&rdquo; approach is well suited when the maintainer wants out ASAP, or for
smaller projects with little activity. But, for active projects with a patient
maintainer, the hand-off approach is less disruptive. Start talking with some of
your major contributors about <a href="https://drewdevault.com/2018/06/01/How-I-maintain-FOSS-projects.html">increasing their involvement</a>
in the administrative side of the projects. Mentor them on doing code reviews,
ticket triage, sysadmin stuff, marketing - all the stuff you have to do - and
gradually share these responsibilities with them.  These people eventually
become productive co-maintainers, and once established you can step away from
the project with little fanfare.</p>
<p>Taking this approach can also help you find healthier ways to be involved in
your own project. This can allow you to focus on the work you enjoy and spend
less time on the work you don&rsquo;t enjoy, which might even restore your enthusiasm
for the project outright! This is also a good idea even if you aren&rsquo;t planning
on stepping down - it encourages your contributors to take personal stake in the
project, which makes them more productive and engaged. This also makes your
community more resilient to <a href="https://tvtropes.org/pmwiki/pmwiki.php/Main/AuthorExistenceFailure">author existence failure</a>, so
that when circumstance forces you to step down the project continues to be
healthy.</p>
<p>It&rsquo;s important to always be happy in your work, and especially in your volunteer
work. If it&rsquo;s not working, then change it. For me, this happens in different
ways. I&rsquo;ve abandoned projects outright and sent users off to make their own fork
before.  I&rsquo;ve also handed projects over to their major contributors. In some
projects I&rsquo;ve appointed new maintainers and scaled back my role to a mere
contributor, and in other projects I&rsquo;ve moved towards roles in marketing,
outreach, management, and stepped away from development. There&rsquo;s no shame in
any of these changes - you still deserve pride in your accomplishments, and
seeking constructive solutions to burnout would do your community a great
service.</p>


guid: https://drewdevault.com/2018/12/20/Porting-Alpine-Linux-to-RISC-V.html
title: Porting Alpine Linux to RISC-V
url: https://drewdevault.com/2018/12/20/Porting-Alpine-Linux-to-RISC-V.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I recently received my <a href="https://www.sifive.com/boards/hifive-unleashed">HiFive Unleashed</a>, after several excruciating
months of waiting, and it&rsquo;s incredibly cool. For those unaware, the HiFive
Unleashed is the first consumer-facing Linux-capable <a href="https://en.wikipedia.org/wiki/RISC-V">RISC-V</a> hardware.
For anyone who&rsquo;s still lost, RISC-V is an <a href="https://github.com/riscv">open</a>,
royalty-free <a href="https://en.wikipedia.org/wiki/Instruction_set_architecture">instruction set
architecture</a>, and
the HiFive is an <a href="https://github.com/sifive">open</a> CPU implementing it. And here
it is on my dining room table:</p>
<p><img src="https://sr.ht/JMao.jpg" alt=""></p>
<p>This board is <em>cool</em>. I&rsquo;m working on making this hardware available to
<a href="https://meta.sr.ht">builds.sr.ht</a> users in the next few months, where I intend to use it to
automate the remainder of the Alpine Linux port and make it available to any
other operating systems (including non-Linux) and userspace software which are
interested in working on a RISC-V port. I&rsquo;m fairly certain that this will be the
first time hardware-backed RISC-V cycles are being made available to the public.</p>
<p>There are two phases to porting an operating system to a new architecture:
bootstrapping and, uh, porting. For lack of a better term. As part of
bootstrapping, you need to obtain a cross-compiler, port libc, and cross-compile
the basics. Bootstrapping ends once the system is <em>self-hosting</em>: able to
compile itself. The &ldquo;porting&rdquo; process involves compiling all of the packages
available for your operating system, which can take a long time and is generally
automated.</p>
<p>The first order of business is the cross-compiler. RISC-V support landed in
binutils 2.28 and gcc 7.1 several releases ago, so no need to worry about adding
a RISC-V target to our compiler. Building both with
<code>--target=riscv64-linux-musl</code> is sufficient to complete this step. The other
major piece is the C standard library, or libc. Unlike the C compiler, this step
required some extra effort on my part - the RISC-V port of musl libc, which
Alpine Linux is based on, is a work-in-progress and has not yet been upstreamed.</p>
<p>There does exist <a href="https://github.com/riscv/riscv-musl">a patch</a> for RISC-V support, though it had never
been tested at a scale like this. Accordingly, I ran into several bugs, for
which I wrote several patches (<a href="https://github.com/riscv/riscv-musl/pull/2">1</a> <a href="https://github.com/riscv/riscv-musl/pull/3">2</a> <a href="https://github.com/riscv/riscv-musl/pull/4">3</a>). Having a working distro
based on the RISC-V port makes a much more compelling argument for the maturity
of the port, and for its inclusion upstream, so I&rsquo;m happy to have caught these
issues. Until then, I added the port and my patches to the Alpine Linux musl
package manually.</p>
<p>A C compiler and libc implementation open the floodgates to porting a huge
volume of software to your platform. The next step is to identify and port the
essential packages for a self-hosting system.  For this, Alpine has a great
<a href="https://git.alpinelinux.org/cgit/aports/tree/scripts/bootstrap.sh">bootstrapping script</a> which handles preparing the cross-compiler
and building the base system. Many (if not most) of these packages required
patching, tweaks, and manual intervention - this isn&rsquo;t a turnkey solution - but
it is an incredibly useful tool. The most important packages at this step are
the native toolchain<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>, the package manager itself, and various other useful
things like tar, patch, openssl, and so on.</p>
<p>Once the essential packages are built and the system can compile itself, the
long porting process begins. It&rsquo;s generally wise to drop the cross-compiler here
and start doing native builds, if your hardware is fast enough. This is a
tradeoff, because the RISC-V system is somewhat slower than my x86_64 bootstrap
machine - but many packages require lots of manual tweaks and patching to get
cross-compiling working. The time saved by not worrying about this makes up for
the slower build times<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>.</p>
<p>There are thousands of packages, so the next step for me (and anyone else
working on a port) is to automate the remainder of the process. For me, an
intermediate step is integrating this with builds.sr.ht to organize my own work
and to make cycles available to other people interested in RISC-V. Not all
packages are going to be ported for free - but many will! Once you unlock the
programming languages - C, Python, Perl, Ruby<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>, etc - most open source
software is pretty portable across architectures. One of my core goals with
sr.ht is to encourage portable software to proliferate!</p>
<p>If any readers have their own RISC-V hardware, or want to try it with qemu, I
have a RISC-V Alpine Linux repository here<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>. Something like this will install
it to /mnt:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">apk add <span class="se">\
</span><span class="se"></span>    -X https://mirror.sr.ht/alpine/main/ <span class="se">\
</span><span class="se"></span>    --allow-untrusted <span class="se">\
</span><span class="se"></span>    --arch<span class="o">=</span>riscv64 <span class="se">\
</span><span class="se"></span>    --root<span class="o">=</span>/mnt <span class="se">\
</span><span class="se"></span>    alpine-base alpine-sdk vim chrony
</code></pre></div><p>Run <code>/bin/busybox --install</code> and <code>apk fix</code> on first boot. This is still a work
in progress, so configuring the rest is an exercise left to the reader until I
can clean up the process and make a nice install script. Good luck!</p>
<hr>
<p>Closing note: big thanks to the help from the community in #riscv on Freenode,
and to the hard work of the Debian and Fedora teams paving a lot of the way and
getting patches out there for lots of software! I still got to have all the fun
working on musl so I wasn&rsquo;t entirely on the beaten path :)</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Meaning a compiler which both <em>targets</em> RISC-V and <em>runs</em> on RISC-V. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>I was actually really impressed with the speed of the HiFive Unleashed. The main bottleneck is the mmcblk driver - once you get files in the kernel cache things are quite pleasant and snappy. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>I have all four of these now! <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p><a href="https://mirror.sr.ht/alpine/main/">main</a>, <a href="https://mirror.sr.ht/alpine/community/">community</a>, <a href="https://mirror.sr.ht/alpine/testing/">testing</a> <a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2018/12/28/Anatomy-of-a-shell.html
title: Anatomy of a shell
url: https://drewdevault.com/2018/12/28/Anatomy-of-a-shell.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I&rsquo;ve been contributing where I can to Simon Ser&rsquo;s <a href="https://git.sr.ht/~emersion/mrsh">mrsh</a> project, a
work-in-progress strictly POSIX shell implementation. I worked on some small
mrsh features during my holiday travels and it&rsquo;s in the forefront of my mind, so
I&rsquo;d like to share some of its design details with you.</p>
<p>There are two main components to a shell: parsing and execution. mrsh uses a
simple <a href="https://en.wikipedia.org/wiki/Recursive_descent_parser">recursive descent parser</a> to generate an AST (Abstract Syntax
Tree, or an in-memory model of the structure of the parsed source). This design
was chosen to simplify the code and avoid dependencies like flex/bison, and is a
good choice given that performance isn&rsquo;t critical for parsing shell scripts.
Here&rsquo;s an example of the input source and output AST:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh"><span class="cp">#!/bin/sh
</span><span class="cp"></span>say_hello<span class="o">()</span> <span class="o">{</span>
	<span class="nb">echo</span> <span class="s2">&#34;hello </span><span class="nv">$1</span><span class="s2">!&#34;</span>
<span class="o">}</span>

<span class="nv">who</span><span class="o">=</span><span class="k">$(</span>whoami<span class="k">)</span>
say_hello <span class="s2">&#34;</span><span class="nv">$who</span><span class="s2">&#34;</span>
</code></pre></div><p>This script is parsed into this AST (this is the output of <code>mrsh -n test.sh</code>):</p>
<pre><code>program
program
└─command_list ─ pipeline
  └─function_definition say_hello ─ brace_group
    └─command_list ─ pipeline
      └─simple_command
        ├─name ─ word_string [3:2 → 3:6] echo
        └─argument 1 ─ word_list (quoted)
          ├─word_string [3:8 → 3:14] hello
          ├─word_parameter
          │ └─name 1
          └─word_string [3:16 → 3:17] !
program
program
└─command_list ─ pipeline
  └─simple_command
    └─assignment
      ├─name who
      └─value ─ word_command ─ program
        └─command_list ─ pipeline
          └─simple_command
            └─name ─ word_string [6:7 → 6:13] whoami
program
└─command_list ─ pipeline
  └─simple_command
    ├─name ─ word_string [7:1 → 7:10] say_hello
    └─argument 1 ─ word_list (quoted)
      └─word_parameter
        └─name who
</code></pre><p>Most of these names come directly from the <a href="http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html">POSIX shell specification</a>.
The parser and AST is made available as a standalone public interface of
libmrsh, which can be used for a variety of use-cases like syntax-aware text
editors, syntax highlighting (see <a href="https://git.sr.ht/~emersion/mrsh/tree/master/highlight.c"><code>highlight.c</code></a>), linters, etc. The most
important use-case is, of course, task planning and execution.</p>
<p>Most of these AST nodes becomes a <em>task</em>. A task defines an implementation of
the following interface:</p>
<div class="highlight"><pre class="chroma"><code class="language-c" data-lang="c"><span class="k">struct</span> <span class="n">task_interface</span> <span class="p">{</span>
	<span class="cm">/**
</span><span class="cm">	 * Request a status update from the task. This starts or continues it.
</span><span class="cm">	 * `poll` must return without blocking with the current task&#39;s status:
</span><span class="cm">	 *
</span><span class="cm">	 * - TASK_STATUS_WAIT in case the task is pending
</span><span class="cm">	 * - TASK_STATUS_ERROR in case a fatal error occured
</span><span class="cm">	 * - A positive (or null) code in case the task finished
</span><span class="cm">	 *
</span><span class="cm">	 * `poll` will be called over and over until the task goes out of the
</span><span class="cm">	 * TASK_STATUS_WAIT state. Once the task is no longer in progress, the
</span><span class="cm">	 * returned state is cached and `poll` won&#39;t be called anymore.
</span><span class="cm">	 */</span>
	<span class="kt">int</span> <span class="p">(</span><span class="o">*</span><span class="n">poll</span><span class="p">)(</span><span class="k">struct</span> <span class="n">task</span> <span class="o">*</span><span class="n">task</span><span class="p">,</span> <span class="k">struct</span> <span class="n">context</span> <span class="o">*</span><span class="n">ctx</span><span class="p">);</span>
	<span class="kt">void</span> <span class="p">(</span><span class="o">*</span><span class="n">destroy</span><span class="p">)(</span><span class="k">struct</span> <span class="n">task</span> <span class="o">*</span><span class="n">task</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div><p>Most of the time the task will just do its thing. Many tasks will have sub-tasks
as well, such as a command list executing a list of commands, or each branch of
an if statement, which it can defer to with <code>task_poll</code>. Many tasks will wait on
an external process, in which case it can return TASK_STATUS_WAIT to have the
process <code>wait</code>ed on. Feel free to browse the <a href="https://git.sr.ht/~emersion/mrsh/tree/master/shell/task">full list of tasks</a> to get
an idea.</p>
<p>One concern more specific to POSIX shells is built-in commands. Some commands
have to be built-in because they manipulate the shell&rsquo;s state, such as <code>.</code> and
<code>cd</code>. Others, like <code>true</code> &amp; <code>false</code>, are there for performance reasons, since
they&rsquo;re simple and easily implemented internally. POSIX specifies <a href="http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14">a list of
special builtins</a> which are necessary to implement in the shell
itself. There&rsquo;s <a href="http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_01_01">a second list</a> that must be present for the shell
environment to be considered POSIX compatible (plus some reserved names like
<code>local</code> and <code>pushd</code> that invoke undefined behavior - mrsh aborts on these).</p>
<p>Here are some links to more interesting parts of the code so you can explore on
your own:</p>
<ul>
<li><a href="https://git.sr.ht/~emersion/mrsh/tree/master/shell/redir.c">Redirection</a> &amp; <a href="https://git.sr.ht/~emersion/mrsh/tree/master/shell/task/pipeline.c">pipelines</a></li>
<li><a href="https://git.sr.ht/~emersion/mrsh/tree/master/shell/task/function_definition.c">Function definition</a> &amp; <a href="https://git.sr.ht/~emersion/mrsh/tree/master/shell/task/command_function.c">execution</a></li>
<li><a href="https://git.sr.ht/~emersion/mrsh/tree/master/builtin/dot.c">The . builtin</a></li>
<li><a href="https://git.sr.ht/~emersion/mrsh/tree/master/main.c">main.c and the REPL</a></li>
</ul>
<p>I might write more articles in the future diving into specific concepts, feel
free to shoot me an email if you have suggestions. Shoutout to Simon for
building such a cool project! I&rsquo;m looking forward to contributing more until we
have a really nice strictly POSIX shell.</p>


guid: https://drewdevault.com/2019/01/01/Patches-welcome.html
title: Patches welcome
url: https://drewdevault.com/2019/01/01/Patches-welcome.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Happy new year! This is always a weird &ldquo;holiday&rdquo; for me, since all of the fun
happened last night. Today is just kind of&hellip; I guess a chance for everyone to
sober up before work tomorrow? It does tend to invite a sense of reflection and
is the ideal time to plan for the year ahead. One of my goals in 2019 is to
change more people&rsquo;s thinking about the open source community and what it means
to count among their number.</p>
<p>I think there&rsquo;s a certain mode of thinking which lends itself to a more
productive free software community and a happier free software contributor. Free
software is not <em>theirs</em> - it&rsquo;s <em>ours</em>. Linux doesn&rsquo;t belong to Linus Torvalds.
Firefox doesn&rsquo;t belong to Mozilla, vim doesn&rsquo;t belong to Bram Moolenaar, and
ffmpeg doesn&rsquo;t belong to Fabrice Bellard. These projects belong to everyone.
That includes you! In this way, we reap the benefits of open source, but we also
shoulder the responsibilities. I&rsquo;m not referring to some abstract sense of
reponsibility, but the tangible ones, like fixing bugs or developing new
features.</p>
<p>One of the great things about this community is how easy it is to release your
software under a FOSS license. You have no obligations to the software once it&rsquo;s
released, except the obligations you hold yourself to (i.e. &ldquo;if this software
makes my computer work, and I want to use my computer, I need to keep this
software in good working order&rdquo;). It&rsquo;s important for users to remember that
they&rsquo;re not entitled to anything other than the rights laid out in the license,
too. You&rsquo;re not entitled to bug fixes or new features - you&rsquo;re <em>empowered</em> by
free software to make those changes yourself.</p>
<p>Sometimes, when working on sway, someone says something like &ldquo;oh, it&rsquo;s a bug in
libwayland&rdquo;. My response is generally along the lines of &ldquo;I guess you&rsquo;re writing
a libwayland patch then!&rdquo; The goal hasn&rsquo;t changed, only the route. It&rsquo;s no
different from being in the weeds and realizing you need to do some refactoring
first. If a problem in some FOSS project, be it a bug or a conspicuously missing
feature, is in the way of your goals, it&rsquo;s <em>your problem</em>.  A friend of mine
recently said of a missing feature in a project they have nothing to do with:
&ldquo;adding FreeBSD 12 support is not yet done, but it&rsquo;s on my todo list.&rdquo; I thought
that perfectly embodied the right way to think about FOSS.</p>
<p>When applying this philosophy, you may occasionally have to deal with an
absentee maintainer or a big old pile of legacy spaghetti code. Fork it! Rewrite
it! These are tough marbles but they&rsquo;re the marbles you&rsquo;ve gotta deal with. It&rsquo;s
not as hard as it looks.</p>
<p>The entire world of free software is your oyster. Nothing is off-limits: if it&rsquo;s
FOSS, you can work on it. Try not to be intimidated by unknown programming
languages, unfamiliar codebases, or a lack of time. You&rsquo;ll pick up the new
language sooner than you think<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>, all projects are similar enough when you get
down to it, and small amounts of work done infrequently adds up over a long
enough time period.  FOSS doesn&rsquo;t have to move quickly, it just has to keep
moving. The Dawn spacecraft accelerated at 0.003 cm/s<sup>2</sup> and made it to
<a href="https://upload.wikimedia.org/wikipedia/commons/a/a1/PIA19547-Ceres-DwarfPlanet-Dawn-RC3-AnimationFrame25-20150504.jpg">another world</a><sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Especially if you have a reason to learn it, like this bug you need to fix <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Actually, it visited 3. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/01/13/Backups-and-redundancy-at-sr.ht.html
title: Backups & redundancy at sr.ht
url: https://drewdevault.com/2019/01/13/Backups-and-redundancy-at-sr.ht.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p><a href="https://sr.ht">sr.ht</a><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> is <a href="https://git.sr.ht/~sircmpwn?search=sr.ht">100% open source</a> and I encourage
people to install it on their own infrastructure, especially if they&rsquo;ll be
sending patches upstream. However, I am equally thrilled to host sr.ht for you
on the &ldquo;official&rdquo; instance, and most users find this useful because the
maintenance burden is non-trivial. Today I&rsquo;ll give you an idea of what your
subscription fee pays for. In this first post on ops at sr.ht, I&rsquo;ll talk about
backups and redundancy. In future posts, I&rsquo;ll talk about security, high
availability, automation, and more.</p>
<p>As sr.ht is still in the alpha phase, high availability has been on the
backburner. However, data integrity has always been of paramount importance to
me. The very earliest versions of sr.ht, from well before it was even trying to
be a software forge, made a point to never lose a single byte of user data.
Outages are okay - so long as when service is restored, everything is still
there. Over time I&rsquo;m working to make outages a thing of the past, too, but let&rsquo;s
start with backups.</p>
<p>There are several ways that sr.ht stores data:</p>
<ul>
<li>Important data on the filesystem (e.g. bare git repositories)</li>
<li>Important persistent data in PostgreSQL</li>
<li>Unimportant ephemeral data in Redis (&amp; caches)</li>
<li>Miscellaneous filesystem storage, like the operating system</li>
</ul>
<p>Some of this data is important and kept redundant (PostgreSQL, git repos), and
others are unimportant and is not redundant. For example, I store a rendered
Markdown cache for git.sr.ht in Redis. If the Redis cluster goes <em>poof</em>, the
source Markdown is still available, so I don&rsquo;t bother backing up Redis. Most
services run in a VM and I generally don&rsquo;t store important data on these - the
hosts usually only have one hard drive with no backups and no redundancy. If the
host dies, I have to reprovision all of those VMs.</p>
<p>Other data is more important. Consider PostgreSQL, which contains some of the
most important data for sr.ht. I have one master PostgreSQL server, a dedicated
server in the space I colocate in my home town of Philadelphia. I run sr.ht on
this server, but I also use it for a variety of other projects - I maintain many
myself, and I volunteer as a sysadmin for more still. This box (named Remilia)
has four hard drives configured in a ZRAID (ZFS). I buy these hard drives from a
variety of vendors, mostly Western Digital and Seagate, and from different
batches - reducing the likelihood that they&rsquo;ll fail around the same time. ZFS is
well-known for it&rsquo;s excellent design, featureset and for simply keeping your
data intact, and I don&rsquo;t trust any other filesystem with important data. I take
ZFS snapshots every 15 minutes and retain them for 30 days. These snapshots are
important for correcting the &ldquo;oh shit, I rm&rsquo;d something important&rdquo; mistakes -
you can mount them later and see what the filesystem looked like at the time
they were taken.</p>
<p>On top of this, the PostgreSQL server is set up with two additional important
features: continuous archiving and streaming replication. Continuous archiving
has PostgreSQL writing each transaction to log files on disk, which represents a
re-playable history of the entire database, and allows you to restore the
database to any point in time. This helps with &ldquo;oh shit, I dropped an important
table&rdquo; mistakes. Streaming replication ships changes to an off-site standby
server, in this case set up in my second colocation in San Francisco (the main
backup box, which we&rsquo;ll talk about more shortly). This takes a near real-time
backup of the database, and has the advantage of being able to quickly failover
to it as the primary database during maintenance and outages (more on this
during the upcoming high availability article). Soon I&rsquo;ll be setting up a second
failover server as well, on-site.</p>
<p>So there are multiple layers to this:</p>
<ul>
<li>ZFS &amp; zraid prevents disk failure from causing data loss</li>
<li>ZFS snapshots allows retrieving filesystem-level data from the past</li>
<li>Continuous archiving allows retrieving database-level data from the past</li>
<li>Streaming replication prevents datacenter existence failure from causing data
loss</li>
</ul>
<p>Having multiple layers of data redundancy here protects sr.ht from a wide
variety of failure modes, and also protects each redundant system from itself -
if any of these systems fails, there&rsquo;s another place to get this data from.</p>
<p>The off-site backup in San Francisco (this box is called Konpaku) has a whopping
52T of storage in two ZFS pools, named &ldquo;small&rdquo; (4T) and &ldquo;large&rdquo; (48T). The
PostgreSQL standby server lives in the small pool, and <a href="https://www.borgbackup.org/">borg
backups</a> live in the large pool. This has the same
ZFS snapshotting and retention policy as Remilia, and also has drives sourced
from a variety of vendors and batches. Borg is how important filesystem-level
data is backed up, for example git repositories on git.sr.ht. Borg is nice
enough to compress, encrypt, and deduplicate its backups for us, which I take
hourly with a cronjob on the machines which own that data. The retention policy
is hourly backups stored for 48 hours, daily backups for 2 weeks, and weekly
backups stored indefinitely.</p>
<p>There are two other crucial steps in maintaining a working backup system:
monitoring and testing. The old wisdom is &ldquo;you don&rsquo;t have backups until you&rsquo;ve
tested them&rdquo;. The simplest monitoring comes from cron - when I provision a new
box, I make sure to set <code>MAILTO</code>, make sure sendmail works, and set up a
deliberately failing cron entry to ensure I hear about it when it breaks. I also
set up zfs-zed to email me whenever ZFS encounters issues, which also has a test
mode you should use. For testing, I periodically provision private replicas of
sr.ht services from backups and make sure that they work as expected. PostgreSQL
replication is fairly new to my setup, but my intention is to switch the primary
and standby servers on every database upgrade for HA<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> purposes, which
conveniently also tests that each standby is up-to-date and still replicating.</p>
<p>To many veteran sysadmins, a lot of this is basic stuff, but it took me a long
time to learn how all of this worked and establish a set of best practices for
myself. With the rise in popularity of managed ops like AWS and GCP, it seems
like ops &amp; sysadmin roles are becoming less common. Some of us still love the
sound of a datacenter and the greater level of control you have over your
services, and as a bonus my users aren&rsquo;t worrying about $bigcorp having access
to their data.</p>
<p>The next ops thing on my todo list is high availability, which is still
in-progress on sr.ht. When it&rsquo;s done, expect another blog post!</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>sr.ht is a software project hosting website, with git hosting, ticket tracking, continuous integration, mailing lists, and more. <a href="https://sr.ht">Try it out!</a> <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>High availability <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/01/15/Im-doing-FOSS-full-time.html
title: I'm going to work full-time on free software
url: https://drewdevault.com/2019/01/15/Im-doing-FOSS-full-time.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Sorry for posting two articles so close to each other - but this is important!
As I&rsquo;m certain many of you know, I maintain a large collection of free software
projects, including <a href="https://github.com/swaywm/sway">sway</a>, <a href="https://github.com/swaywm/wlroots">wlroots</a>, <a href="https://sr.ht">sr.ht</a>,
<a href="https://git.sr.ht/~sircmpwn/scdoc">scdoc</a>, <a href="https://git.sr.ht/~sircmpwn/aerc2">aerc</a>, and <a href="https://git.sr.ht/~sircmpwn">many</a>, <a href="https://github.com/ddevault">many</a>
more. I contribute to more still, working on projects like <a href="https://alpinelinux.org">Alpine
Linux</a>, <a href="https://git.sr.ht/~emersion/mrsh">mrsh</a>, <a href="https://musl-libc.org">musl libc</a>, and anything else I can. Until
now, I&rsquo;ve been working on these in my spare time, but just under a year ago I
wrote &ldquo;<a href="https://drewdevault.com/2018/02/24/The-road-to-sustainable-FOSS.html">The path to sustainably working on FOSS full-time</a>&rdquo; laying out
my future plans.  Today I&rsquo;m proud to tell you that, thanks to everyone&rsquo;s
support, I&rsquo;ll be working on free software full-time starting in February.</p>
<p>I&rsquo;m so excited! I owe many people a great deal of thanks. To everyone who has
donated to my fosspay, Patreon, and LiberaPay accounts: thank you. To all of the
sr.ht users who chose to pay for their account despite it being an alpha: thank
you. I also owe a thanks to all of the amazing contributors who give their spare
time towards making the projects I maintain even better, without whom my
software wouldn&rsquo;t be anywhere near as useful to anyone.</p>
<p>I don&rsquo;t want to make grandiose promises right away, but I&rsquo;m confident that
increasing my commitment to open source to this degree is going to have a major
impact on my projects. For now, my primary focus is sr.ht: its paid users make
up the majority of the funding. Relatedly, I plan to invest more time in Alpine
Linux on RISC-V and making RISC-V builds available to the sr.ht community.  Sway
and wlroots are in good shape as we quickly approach sway 1.0, and for this
reason I want to give a higher priority to my smaller, more neglected projects
like aerc for the time being. As I learn more about my bandwidth under these new
conditions, I&rsquo;ll expand my plans accordingly.</p>
<p>I need to clarify that despite choosing to work full-time on these projects, my
income is going to be negative for a while. I have enough savings and income now
that I feel comfortable making the leap, and I plan on working my ass off before
my runway ends to earn the additional subscriptions to sr.ht and donations to
fosspay et al that will make this decision sustainable in the long term. If that
doesn&rsquo;t happen before I get near the end of my runway, I&rsquo;ll have to start
looking for different work again. I&rsquo;m depending on your continued support. If
you appreciate my work but haven&rsquo;t yet, please consider <a href="https://meta.sr.ht/billing/initial">buying a subscription
to sr.ht</a> or <a href="https://drewdevault.com/donate">donating to my general
projects fund</a>. Thank you!</p>
<p>All said, words cannot describe how excited I am. It&rsquo;s been my dream for years
to work on these projects full-time. Hitting this critical threshold of funding
allows me to tremendously accelerate the progress of these projects. If you were
impressed by what I built in my spare time, just wait until you see what we can
accomplish now!</p>
<p><img src="https://sr.ht/YsHI.png" alt=""></p>
<p>From the bottom of my heart, thank you for your support!</p>
<p>P.S: I&rsquo;ll see you at <a href="https://fosdem.org/2019/">FOSDEM</a>! Ask me for a sticker.</p>


guid: https://drewdevault.com/2019/01/23/Why-I-use-old-hardware.html
title: Why I use old hardware
url: https://drewdevault.com/2019/01/23/Why-I-use-old-hardware.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Recently I was making sure my main laptop is ready for travel<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>, which mostly
just entails syncing up the latest version of my music collection. This laptop
is a Thinkpad X200, which turns 11 years old in July and is my main workstation
away from home (though I bring a second monitor and an external keyboard for
long trips). This laptop is a great piece of hardware. 100% of the hardware is
supported by the upstream Linux kernel, including the usual offenders like WiFi
and Bluetooth. Niche operating systems like 9front and Minix work great, too.
Even coreboot works! It&rsquo;s durable, user-serviceable, light, and still looks
brand new after all of these years. I love all of these things, but there&rsquo;s no
denying that it&rsquo;s 11 years behind on performance innovations.</p>
<p>Last year <a href="https://kde.org">KDE</a> generously <a href="https://drewdevault.com/2018/04/28/KDE-Sprint-retrospective.html">invited me</a> to and
sponsored my travel to their development sprint in Berlin. One of my friends
there teased me - in a friendly way - about my laptop, asking why I used such an
old system. There was a pensive moment when I answered: &ldquo;it forces me to
empathise with users who can&rsquo;t use high-end hardware&rdquo;. I showed him how it could
cold boot to a productive <a href="https://swaywm.org">sway</a> desktop in &lt;30 seconds,
then I installed KDE to compare. It doubled the amount of disk space in use,
took almost 10x as long to reach a usable desktop, and had severe rendering
issues with my old Intel GPU.</p>
<p>To be clear, KDE is a wonderful piece of software and my first recommendation to
most non-technical computer users who ask me for advice on using Linux. But
software often grows to use the hardware you give it. Software developers tend
to be computer enthusiasts, and use enthusiast-grade hardware. In reality, this
high-end hardware isn&rsquo;t really <em>necessary</em> for most applications outside of
video encoding, machine learning, and a few other domains.</p>
<p>I do have a more powerful workstation at home, but it&rsquo;s not really anything
special. I upgrade it very infrequently. I bought a new mid-range GPU which is
able to drive my four displays<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> last year, I&rsquo;ve added the occasional hard
drive as it gets full, and I replaced the case with something lighter weight 3
years ago. Outside of those minor upgrades, I&rsquo;ve been using the same desktop
workstation for 7 years, and intend to use it for much longer. My servers are
similarly running on older hardware which is spec&rsquo;d to their needs (actually, I
left a lot of room to grow and <em>still</em> was able to buy old hardware).</p>
<p>My 11-year-old laptop can compile the Linux kernel from scratch in 20 minutes,
and it can play 1080p video in real-time. That&rsquo;s all I need! Many users cannot
afford high-end computer hardware, and most have better things to spend their
money on. And you know, I work hard for my money, too - if I can get a computer
which can do nearly 5 <em>billion</em> operations per second for $60, that should be
sufficient to solve nearly any problem. No doubt, there are faster laptops out
there, many of them with similarly impressive levels of compatibility with my
ideals. But why bother?</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>To <a href="https://fosdem.org/2019/">FOSDEM</a> - see you there! <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>I have a variety of displays and display configurations for the purpose of continuously testing sway/wlroots in those situations <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/01/30/Why-I-built-sr.ht-with-Flask.html
title: Why I chose Flask to build sr.ht's mini-services
url: https://drewdevault.com/2019/01/30/Why-I-built-sr.ht-with-Flask.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p><a href="https://sr.ht">sr.ht</a> is a large, production-scale suite of web applications (I
call them &ldquo;mini-services&rdquo;, as they strike a balance between microservices and
monolithic applications) which are built in Python with
<a href="http://flask.pocoo.org/">Flask</a>. David Lord, one of the maintainers of Flask,
reached out to me when he heard about sr.ht and saw that it was built with
Flask. At his urging, I&rsquo;d like to share the rationale behind the decision and
how it&rsquo;s turned out in the long run.</p>
<p>I have a long history of writing web applications with Flask, so much so that I
think I&rsquo;ve lost count of them by now - at least 15, if not 20. Flask&rsquo;s
simplicity and flexibility is what keeps bringing me back. Frameworks like
Django or Rails are much different: they are the kitchen sink, and then some. I
generally don&rsquo;t need the whole kitchen sink, and if I were given it, I would
want to change some details. Flask is nice because it gives you the basics and
lets you build what you need on top of it, and you&rsquo;re never working around a
cookie-cutter system which doesn&rsquo;t cut your cookies in quite the way you need.</p>
<p>In sr.ht&rsquo;s case in particular, though, I have chosen to extend Flask with <a href="https://git.sr.ht/~sircmpwn/core.sr.ht">a new
module</a> common to all sr.ht projects. After all, each service of
sr.ht has a lot in common with the rest. Some of the things that live in this
core module are:</p>
<ul>
<li>Shared jinja2 templates and stylesheets</li>
<li>Shared rigging for <a href="https://www.sqlalchemy.org/">SQLAlchemy</a> (ORM)</li>
<li>Shared rigging for <a href="https://alembic.sqlalchemy.org/en/latest/">Alembic</a></li>
<li><a href="https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/validation.py">A little validation module</a> I&rsquo;m very proud of</li>
<li>API behavior, webhooks, OAuth, etc, which are consistent throughout sr.ht</li>
</ul>
<p>The mini-service-oriented architecture allows sr.ht services to be deployed
ala-carte for users who only need a fraction of what we offer. This design
requires a lot of custom code to integrate all of the services with each other -
for example, all of the services use a single shared config file, which contains
both shared config options and service-specific configuration. sr.ht also uses a
novel approach to authentication, in which both user logins and API
authentication is delegated to an external service, meta.sr.ht, requiring
further custom code still. core.sr.ht additionally provides common SQLAlchemy
mixins for things like user tables, which have many common properties, but for
each service may have service-specific columns as well.</p>
<p>Django provides their own ORM, their own authentication, their own models, and
more. In order to meet the design constraints of sr.ht, I&rsquo;d have spent twice as
long ripping out the rest of Django&rsquo;s bits and fixing anything that broke in the
resulting mess. With Flask, these bits were never written for me in the first
place, which gives me the freedom to implement this design greenfield. Flask is
small and what code it does bring to the table is highly pluggable.</p>
<p>Though it&rsquo;s well suited to many of my needs, I don&rsquo;t think Flask is perfect. A
few things I dislike about it:</p>
<ul>
<li>First-class <a href="http://jinja.pocoo.org/">jinja2</a> support is probably out of
scope.</li>
<li>flask.Flask and flask.Blueprint should be the same thing.</li>
<li>I&rsquo;m not a fan of Flask&rsquo;s approach to configuration. I have a better(?) config
module that I drag around to all of my projects.</li>
</ul>
<p>And to summarize the good:</p>
<ul>
<li>It provides a nice no-nonsense interface for requests, responses, and routing.</li>
<li>It has a lot of nice hooks for adding your own middleware.</li>
<li>It doesn&rsquo;t do much more than that, which means you&rsquo;re free to choose and
compose other tools to make up the difference.</li>
</ul>
<p>I think that on the whole it&rsquo;s quite good. There are frameworks which are
smaller still - but I think Flask hits a sweet spot. If you&rsquo;re making a
monolithic web app and can live within the on-rails Django experience, you might
want to use it. But if you are making smaller apps or need to rig things up in a
unique way - something I find myself doing almost every time - Flask is probably
for you.</p>


guid: https://drewdevault.com/2019/02/05/FOSDEM-recap.html
title: My experiences at FOSDEM 2019
url: https://drewdevault.com/2019/02/05/FOSDEM-recap.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Currently in a plane on my way home from FOSDEM and, as seems to be a recurring
pattern when I fly long distances home after attending a conference, a recap is
readily flowing from my fingertips. This was my first year at FOSDEM, and I&rsquo;m
glad that I came. I&rsquo;m already excited for next year! It was also my first year
volunteering, which was equally great and another thing I expect to repeat.</p>
<p>My biggest feeling during the event was one of incredible business. My
scatterbrained interests throughout the domain of free software came back to
haunt me as I struggled to keep up with all of the people I had to meet &amp; thank,
all of the sessions I wanted to see, and all of the dinners &amp; outings I wanted
to attend. Before all of the fuss, though, I was lucky enough to have a day and
a half to myself (and later with <a href="https://emersion.fr">Simon Ser</a>) to enjoy
Brussels with.</p>
<p>The first FOSDEM-related event I found myself was when the Arch Linux developers
graciously invited me to their dinner on Friday. I have a long friendship with
several Arch developers, but never met any in person. We were speaking in the
weeks before FOSDEM about how to save them from their subversion nightmare, and
we spoke a little bit about some ideas for fixing this, but mostly we just had a
good time and got to know each other better. Later in the week, Jerome finally
convinced me to apply to become an Arch Trusted User, and in the coming months I
hope to work with them on a nice next-generation system for Arch Linux package
maintenance.</p>
<p>The hallway track<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> continued to be the highlight of the event. Later Friday
night, I had volunteered to staff the FOSDEM beer event&rsquo;s late shift, so the
inevitability of time and biology led to missing the first half of day one. I
ended up wiggling my way into the BSD room and saw a cool talk on NetBSD - long
one of my favorites among the BSDs, and learned that the speaker had a cool
project which will save me a lot of time when adding NetBSD support to sr.ht.
Grabbed his email afterwards and met up with my friends from KDE for lunch. We
met up with Daniel Stone as well, and spoke for a while about how we&rsquo;re finally
going to approach unifying and standardizing the Wayland ecosystem. This
discussion took place waiting outside the graphics room for the Pipewire talk.
Simon has been working on a portal to connect sway&rsquo;s Wayland protocols with the
dbus-based ecosystem Pipewire lives in, and along with KDE&rsquo;s Roman Glig they had
some interesting questions for the presenter.</p>
<p>The second day was quite a bit different. My other role as a volunteer was doing
A/V support in the rooms. For this I got a <em>second</em> shirt, with a different
color! I think next year I may try to collect them all. This was interesting and
slow work, and basically entailed walking down to the stage crouched down to
tweak the mic volume until someone on IRC from the war room said it was better.
I did get to observe more exciting crises over IRC from the comfort of my
relatively normal room, though, and got to play a bit with the astonishingly
sophisticated A/V setup FOSDEM uses. After that I grabbed a light lunch and
passed the time by playing Magic: the Gathering with a group we found in the
FOSDEM bar. I grabbed some Club Mates - I love them but they&rsquo;re super difficult
to get in the United States - and waited until the highlight of the event: the
sr.ht and sway meetups.</p>
<p>Big shoutout to the FOSDEM organizers for entertaining our last-minute requests
to have a space to meet users of both groups. The turnout for both rooms was
way more than I expected - almost 50! It seemed like every seat was filled. I
was also surprised at how distinct the groups where, with only a 5-10% overlap.
After making sure everyone got a sticker, there was some really great questions
and feedback from the sr.ht crowd. A particularly interesting tangent had me
defending the email choice to a skeptic and getting a lot of good feedback and
insights from the rest of the room, as well as elaborating on my plans to
improve the workflow for those less comfortable with email. There was naturally
some discussion about the crappy name and my plans to fix it, and I had the
pleasure of demoing the experimental Fedora builds live to someone who was
asking when there would be Fedora support. It was also great to meet many of the
users and contributors who I&rsquo;ve been working with online, and made sure to thank
them in person - particularly Ivan Habunek, a prolific sr.ht contributor who was
part of our roaming sway/sr.ht/Arch Linux/etc clan throughout FOSDEM.</p>
<p>The sway meetup was equally fun, and I thank the attendees for bearing with me
while I answered the post-meetup questions and comments from the sr.ht crowd -
my fault for scheduling two back-to-back sessions. We started off with a bang by
releasing sway 1.0-rc1, then turned to questions and feedback from the crowd.
Simon had a lot to say during the sway meetup as well, explaining his work and
future plans for the project, and together we also explained our somewhat novel
philosophy on project governance that I credit the success of the project to.
It&rsquo;s designed to maximize contributors, and it&rsquo;s entirely to their credit that
the success of sway and wlroots is owed. Speaking of the future of sway and
wlroots, I also met Guido, an engineer at Purism who works with wlroots, again
after our initial meeting at XDC 2018. This time, Guido brought a gift - a
Librem 5 dev board for the wlroots team to use. Thank you! You&rsquo;ll hear more
about our work with this board in the coming months as I use it to improve touch
support for sway and send it out on loan to various wlroots project developers.</p>
<p>I had a flight home Sunday evening so we had a hasty and delicious dinner, a
quick round of beers, and finally parted ways. An overnight in Dublin and here I
am - on the plane home to Philly, with 43% of my battery<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> and an estimated 3
hours left in-flight. FOSDEM was great - a huge thanks to the organizers and
volunteers! I&rsquo;m looking forward to next year.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>The part of the conference which takes place in the hallway, i.e. socializing with other attendees. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Paranoia about which led me to spend some time optimizing my development environment&rsquo;s power consumption a bit <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/02/10/Wayland-misconceptions-debunked.html
title: Wayland misconceptions debunked
url: https://drewdevault.com/2019/02/10/Wayland-misconceptions-debunked.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>This article has been on my backburner for a while, but it seems Wayland FUD is
making the news again recently, so I&rsquo;ve bumped up the priority a bit. For those
new to my blog, I am the maintainer of
<a href="https://github.com/swaywm/wlroots">wlroots</a>, a library which implements much of
the functionality required of a Wayland compositor and is arguably the single
most influential project in Wayland right now; and <a href="https://swaywm.org">sway</a>, a
popular Wayland compositor which is nearing version 1.0. Let&rsquo;s go over some of
the common misconceptions I hear about Wayland and why they&rsquo;re wrong. Feel free
to pick and choose the misconceptions you believe to read and disregard the
rest.</p>
<p>The art of hating Wayland has become a cult affair. We don&rsquo;t need to put
ourselves into camps at war. Please try not to read this article through the
lens of anger.</p>
<h3 id="wayland-isnt-more-secure-look-at-this-keylogger">Wayland isn&rsquo;t more secure, look at this keylogger!</h3>
<p>There is an <a href="https://github.com/Aishou/wayland-keylogger">unfortunate GitHub
project</a> called &ldquo;Wayland keylogger&rdquo;
whose mode of operation is using <code>LD_PRELOAD</code> to intercept calls to the
libwayland shared library and record keypresses from it. The problem with this
&ldquo;critique&rdquo; is stated in the <code>README.md</code> file, though most don&rsquo;t read past the
title of the repository. Wayland is only <em>one part</em> of an otherwise secure
system. Using <code>LD_PRELOAD</code> is effectively equivalent to rewriting client
programs to log keys themselves, and any program which is in a position to do
this has already won. If I rephrased this as &ldquo;Wayland can be keylogged, assuming
the attacker can sneak some evil code into your .bashrc&rdquo;, the obviousness of
this truth should become immediately apparent.</p>
<p>Some people have also told me that they can log keys by opening <code>/dev/input/*</code>
files and reading input events. They&rsquo;re right! Try it yourself: <code>sudo libinput debug-events</code>. The catch should also be immediately obvious: ask
yourself why this needs to be run with <code>sudo</code>.</p>
<h3 id="wayland-doesnt-support-screenshotscapture">Wayland doesn&rsquo;t support screenshots/capture!</h3>
<p>The <a href="https://github.com/wayland-project/wayland/blob/master/protocol/wayland.xml">core Wayland protocol</a> does not define a mechanism for taking
screenshots. Here&rsquo;s another thing it doesn&rsquo;t define: how to open application
windows, like gedit and Firefox. The Wayland protocol is very conservative and
general purpose, and is built with use-cases other than desktop systems in mind.
To this end it only implements the lowest common denominator, and leaves the
rest to protocol extensions. There is a process for defining, implementing,
maturing, and standardizing these extensions, though the last part is in need of
improvements - which are under discussion.</p>
<p>There are two protocols for the purpose of screenshots and screen recording,
which are developed by wlroots and supported by a strong majority of Wayland
compositors: <a href="https://github.com/swaywm/wlroots/blob/master/protocol/wlr-screencopy-unstable-v1.xml">screencopy</a> and
<a href="https://github.com/swaywm/wlroots/blob/master/protocol/wlr-export-dmabuf-unstable-v1.xml">dmabuf-export</a>, respectively for copying pixels (best for
screenshots) and exporting DMA buffers (best for real-time video capture).</p>
<p>There are two approaches to this endorsed by different camps in Wayland: these
Wayland protocols, and a dbus protocol based on Pipewire. Progress is being made
on making these approaches talk to each other via <a href="https://github.com/emersion/xdg-desktop-portal-wlr">xdg-desktop-portal</a>,
which will make just about every client and compositor work together.</p>
<h3 id="wayland-doesnt-have-a-secondary-clipboard">Wayland doesn&rsquo;t have a secondary clipboard!</h3>
<p>Secondary clipboard support (aka primary selection) was first implemented as
<a href="https://github.com/swaywm/wlroots/blob/master/protocol/gtk-primary-selection.xml">gtk-primary-selection</a> and was recently standardized as
<a href="https://github.com/wayland-project/wayland-protocols/blob/master/unstable/primary-selection/primary-selection-unstable-v1.xml">wp-primary-selection</a>. It is supported by nearly all Wayland
compositors and clients.</p>
<h3 id="wayland-doesnt-support-clipboard-managers">Wayland doesn&rsquo;t support clipboard managers!</h3>
<p>See <a href="https://github.com/bugaevc/wl-clipboard">wl-clipboard</a></p>
<h3 id="wayland-isnt-suitable-for-embedded-devices">Wayland isn&rsquo;t suitable for embedded devices!</h3>
<p>Some people argue that Wayland isn&rsquo;t supported on embedded devices or require
proprietary blobs to work. This is <em>very</em> untrue. Firstly, Wayland is a
protocol: the <em>implementations</em> are the ones that need support from drivers, and
a Wayland implementation could be written for basically any driver. You could
implement Wayland by writing Wayland protocol messages on pieces of paper,
passing them to your friend in class, and having them draw your window on their
notebook with a pencil.</p>
<p>That being said, this is also untrue of the implementations. wlroots, which
contains the most popular Wayland rendering backend, implements KMS+DRM+GBM,
which is supported by all open source graphics drivers, and uses GLESv2, which
is the most broadly supported graphics implementation, including on embedded
(which is what the &ldquo;E&rdquo; stands for) and most older hardware. For ancient
hardware, writing an fbdev backend is totally possible and I&rsquo;d merge it in
wlroots if someone put in the time. Writing a more modern KMS+DRM+GBM
implementation for that hardware is equally possible.</p>
<h3 id="wayland-doesnt-have-network-transparency">Wayland doesn&rsquo;t have network transparency!</h3>
<p>This is actually true! But it&rsquo;s not as bad as it&rsquo;s made out to be. Here&rsquo;s why:
X11 forwarding works on Wayland.</p>
<p>Wait, what? Yep: all mainstream desktop Wayland compositors have support for
<strong>Xwayland</strong>, which is an implementation of the X11 server which translates X11
to Wayland, for backwards compatibility. X11 forwarding works with it! So if you
use X11 forwarding on Xorg today, your workflow will work on Wayland unchanged.</p>
<p>However, Wayland itself is not network transparent. The reason for this is that
some protocols rely on file descriptors for transferring information quickly or
in bulk. One example is GPU buffers, so that the Wayland compositor can render
clients without copying data on the GPU - which improves performance
dramatically. However, little about Wayland is inherently network <em>opaque</em>.
Things like sending pixel buffers to the compositor are already abstracted on
Wayland and a network-backed implementation could be easily made. The problem is
that no one seems to really care: all of the people who want network
transparency drank the anti-Wayland kool-aid instead of showing up to put the
work in. If you want to implement this, though, we&rsquo;re here and ready to support
you! Drop by the wlroots <a href="https://webchat.freenode.net/?channels=sway-devel">IRC channel</a> and we&rsquo;re prepared to help
you implement this.</p>
<h3 id="wayland-doesnt-support-remote-desktop">Wayland doesn&rsquo;t support remote desktop!</h3>
<p>This one is also true, but work is ongoing. Several of the pieces are in place:
screen capture and keyboard simulation are there. If an interested developer
wants to add pointer device simulation and tie it all together with librdesktop,
that would be a great boon to the Wayland ecosystem. <a href="https://webchat.freenode.net/?channels=sway-devel">We&rsquo;re waiting to
help!</a></p>
<h3 id="wayland-requires-client-side-decorations">Wayland requires client side decorations!</h3>
<p>This was actually true for a long time, but there was deep contention in the
Wayland ecosystem over this matter. We fought long and hard over this and we now
have a protocol for negotiating client- vs server-side decorations, which is now
fairly broadly supported, including among some of its opponents. You&rsquo;re welcome.</p>
<h3 id="wayland-doesnt-support-hotkey-daemons">Wayland doesn&rsquo;t support hotkey daemons!</h3>
<p>This is a feature, not a bug, but you&rsquo;re free to disagree once you hear the
rationale. There are lots of problems with the idea of hotkey daemons as it
exists on X. What if there&rsquo;s a conflict between several clients who want the
same hotkey? What if the user wants to pick a different hotkey? On top of this,
designing a protocol carefully to avoid keylogging concerns makes it more
difficult still.</p>
<p>To this end, I&rsquo;ve been encouraging client developers who want hotkeys to instead
use some kind of IPC mechanism and a control binary. For example, <code>mako</code>, a
notification daemon, allows you to dismiss notifications by running the <code>makoctl dismiss</code> command. Users are then encouraged to use the compositor&rsquo;s own
keybinding facilities to execute this command. This is more flexible even
outside of keybinding - the user might want to execute this behavior through a
script, too.</p>
<p>Still, if you <em>really</em> want hotkeys, you should start the discussion for
standardizing a protocol. It will be an uphill battle but I believe that a
protocol which addresses everyone&rsquo;s concerns is theoretically possible. <em>You</em>
have to step up, though: no one working on Wayland today seems to care. We are
mostly volunteers working for free in our spare time.</p>
<h3 id="wayland-doesnt-support-nvidia">Wayland doesn&rsquo;t support Nvidia!</h3>
<p>Actually, Nvidia doesn&rsquo;t support us. There are three standard APIs which are
implemented by all graphics drivers in the Linux kernel: DRM (display resource
management), KMS (kernel mode setting), and GBM (generic buffer management). All
three are necessary for most Wayland compositors. Only the first two are
implemented by the Nvidia proprietary driver. In order to support Nvidia,
Wayland compositors need to add code resembling this:</p>
<div class="highlight"><pre class="chroma"><code class="language-c" data-lang="c"><span class="k">if</span> <span class="p">(</span><span class="n">nvidia</span> <span class="n">proprietary</span> <span class="n">driver</span><span class="p">)</span> <span class="p">{</span>
    <span class="cm">/* several thousand lines of code */</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
    <span class="cm">/* several thousand lines of code */</span>
<span class="p">}</span>
</code></pre></div><p>That&rsquo;s terrible! On top of that, we cannot debug the proprietary driver, we
cannot send fixes upstream, and we cannot read the code to understand its
behavior. The mesa code (where much of the important code for many drivers
lives) is a frequent object of study among Wayland compositor developers. We
cannot do this with the proprietary drivers, and it doesn&rsquo;t even implement the
APIs it needs to. They claim to be working on a replacement for GBM which they
hope will satisfy everyone&rsquo;s concerns, but 52 commits in 3 years with over a
year of inactivity isn&rsquo;t a great sign.</p>
<p>To boot, Nvidia is a bad actor on Linux. Compare the talks at FOSDEM 2018
from the <a href="https://archive.fosdem.org/2018/schedule/event/nouveau/">nouveau developers</a> (the open source Nvidia driver) and the
<a href="https://archive.fosdem.org/2018/schedule/event/amd_graphics/">AMDGPU developers</a> (the <em>only</em><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> AMD driver - also open source). The
Nouveau developers discuss all of the ways that Nvidia makes their lives
difficult, up to and including <em>signed firmwares</em>. AMDGPU instead talks about
the process of upstreaming their driver, discuss their new open source Vulkan
driver, and how the community can contribute - and this was presented by paid
AMD staff. I met Intel employees at XDC who were working on a continuous
integration system wherein Intel offers a massive Intel GPU farm to Mesa
developers free-of-charge for working on the open source driver. Nvidia is
clearly a force for bad on the Linux scene and for open source in general, and
the users who reward this by spending oodles of cash on their graphics cards are
not exactly in my good graces.</p>
<p>So in short, people asking for Nvidia proprietary driver support are asking the
wrong people to spend hundreds of hours working for free to write and maintain
an implementation for <em>one</em> driver which represents a harmful force on the Linux
ecosystem and a headache for developers trying to work with it. With respect, my
answer is no.</p>
<h3 id="wayland-doesnt-support-gaming">Wayland doesn&rsquo;t support gaming!</h3>
<p>First-person shooters, among other kinds of games, require &ldquo;locking&rdquo; the pointer
to their window. This requires <a href="https://github.com/wayland-project/wayland-protocols/blob/master/unstable/pointer-constraints/pointer-constraints-unstable-v1.xml">a protocol</a>, which was
standardized in 2015. Adoption has been slower, but it landed in wlroots several
months ago and support was added to sway a few weeks ago.</p>
<h3 id="in-conclusion">In conclusion</h3>
<p>At some point, some of these things have been true. Some have never been true.
It takes time to replace a 30-year incumbent. To be fair, some of these points
are true on GNOME and KDE, but none are inherently problems with the Wayland
protocol. wlroots is a dominating force in the Wayland ecosystem and the tide is
clearly moving our way.</p>
<p>Another thing I want to note is that Xorg still works. If you find your needs
aren&rsquo;t met by Wayland, just keep using X! We won&rsquo;t be offended. I&rsquo;m not trying
to force you to use it. Why you heff to be mad?</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>*actively maintained <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/02/18/Generics-arent-ready-for-Go.html
title: Generics aren't ready for Go
url: https://drewdevault.com/2019/02/18/Generics-arent-ready-for-Go.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>In the distance, a gradual roar begins to grow in volume. A dust cloud is
visible over the horizon. As it nears, the shouts of the oncoming angry mob can
be heard. Suddenly, it stops, and a brief silence ensues. Then the air is filled
with the clackings of hundreds of keyboards, angrily typing the owner&rsquo;s opinion
about generics and Go. The clans of Java, C#, Rust, C++, TypeScript, Haskell,
and more - usually mortal enemies - have combined forces to fight in what may
become one of the greatest flamewars of our time. And none of them read more
than the title of this article before writing their comment.</p>
<p>Have you ever seen someone write something to the effect of &ldquo;I would use Go, but
I need generics&rdquo;? Perhaps we can infer from this that many of the people who are
pining after generics in Go are not, in fact, Go users. Many of them are users
of another programming language that <em>does</em> have generics, and they feel that
generics are a good fit for this language, and therefore a good fit for any
language. The inertia of &ldquo;what I&rsquo;m used to&rdquo; comes to a violent stop when they
try to use Go. People affected by this frustration interpret it as a problem
with Go, that Go is missing some crucial feature - such as generics. But this
lack of features is itself a feature, not a bug.</p>
<p>Go strikes me as one of the most conservative programming languages available
today. It&rsquo;s small and simple, and every detail is carefully thought out. There
are very few dusty corners of Go - in large part because Go has fewer corners in
general than most programming languages. This is a major factor in Go&rsquo;s success
to date, in my opinion. Nearly all of Go&rsquo;s features are bulletproof, and in my
opinion are among the best implementations of their concepts in our entire
industry. Achieving this feat requires having <em>fewer</em> features in total.
Contrast this to C++, which has too many footguns to count. You <em>could</em> write a
book called &ldquo;C++: the good parts&rdquo;, but consider that such a book about Go would
just be a book about Go. There&rsquo;s little room for the bad parts in such a spartan
language.</p>
<p>So how should we innovate in Go? Consider the case of dependency management. Go
1.11 shipped with the first version of Go modules, which, in my opinion, is a
game changer. I passionately hate <code>$GOPATH</code>, and I thought dep wasn&rsquo;t much
better. dep&rsquo;s problem is that it took the dependency management ideas that other
programming languages have been working with and brought the same ideas to Go.
Instead, Go modules took the idea of dependency management and rethought it from
first principles, then landed on a much more elegant solution that I think other
programming languages will spend the next few years catching up with. I like to
make an analogy to physics: dep is like <a href="https://en.wikipedia.org/wiki/General_relativity">General Relativity</a> or <a href="https://en.wikipedia.org/wiki/Standard_Model">the
Standard Model</a>, whereas Go modules are more like the <a href="https://en.wikipedia.org/wiki/Grand_Unified_Theory">Grand Unified
Theory</a>. Go doesn&rsquo;t settle for anything less when adding features. It&rsquo;s
not a language where liberal experimentation with imperfect ideas is desirable.</p>
<p>I feel that this applies to generics. In my opinion, generics are an imperfect
solution to an unsolved problem in computer science. None of the proposals I&rsquo;ve
seen (notably <a href="https://go.googlesource.com/proposal/+/master/design/go2draft-contracts.md">contracts</a>) feel <em>right</em> yet. Some of this is a gut
feeling, but there are tangible problems as well. For example, the space of
problems they solve intersects with other Go features, which weakens the
strength of both features. &ldquo;Which solution do I use to this problem&rdquo; is a
question which different people will answer differently, and consequently their
code at best won&rsquo;t agree on what &ldquo;idiomatic&rdquo; means and at worst will be simply
incompatible.  Another problem is that the proposal changes the meaning of
idiomatic Go in the first place - suddenly huge swaths of the Go code, including
the standard library, will become unidiomatic. One of Go&rsquo;s greatest strengths is
that code written 5 years ago is still idiomatic. It&rsquo;s almost impossible to
write unidiomatic Go code at all.</p>
<p>I used to sneer at the Go maintainers alongside everyone else whenever they&rsquo;d
punt on generics. With so many people pining after it, why haven&rsquo;t they seen
sense yet? How can they know better than all of these people? My tune changed
once I started to use Go more seriously, and now I admire their restraint.  Part
of this is an evolution of my values as a programmer in general: simplicity and
elegance are now the principles I optimize for, even if it means certain classes
of programs are simply not on the table. And I think Go should be comfortable
not being suitable for writing certain classes of programs. I don&rsquo;t think
programming languages should compete with each other in an attempt to become the
perfect solution to every problem.  This is impossible, and attempts will just
create a messy kitchen sink that solves every problem poorly.</p>
<p><img
src="https://sr.ht/TxC_.jpg"
alt="Nina Tucker from Fullmetal Alchemist"
width="320" /></p>
<p class="text-center">
  <small>fig. 1: the result of C++'s attempt to solve all problems</small>
</p>
<p>The constraints imposed by the lack of generics (and other things Go lacks)
breed creativity.  If you&rsquo;re fighting Go&rsquo;s lack of generics trying to do
something Your Way, you might want to step back and consider a solution to the
problem which embraces the limitations of Go instead. Often when I do this the
new solution is a much better design.</p>
<p>So it&rsquo;s my hope that Go will hold out until the right solution presents itself,
and it hasn&rsquo;t yet. Rushing into it to appease the unwashed masses is a bad idea.
There are other good programming languages - use them! I personally use a wide
variety of programming languages, and though I love Go dearly, it probably only
comes in 3rd or 4th place in terms of how frequently it appears in my projects.
It&rsquo;s <em>excellent</em> in its domain and doesn&rsquo;t need to awkwardly stumble into
others.</p>


guid: https://drewdevault.com/2019/02/25/Using-git-with-discipline.html
title: Tips for a disciplined git workflow
url: https://drewdevault.com/2019/02/25/Using-git-with-discipline.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Basic git usage involves typing a few stock commands to &ldquo;<a href="https://xkcd.com/1597/">sync everyone
up</a>&rdquo;. Many people who are frustrated with git become so
because they never progress beyond this surface-level understanding of how it
works. However, mastering git is easily worth your time. How much of your day is
spent using git? I would guess that there are many tools in your belt that you
use half as often and have spent twice the time studying.</p>
<p><a href="https://xkcd.com/1205/"><img src="https://imgs.xkcd.com/comics/is_it_worth_the_time.png" alt=""></a></p>
<p>If you&rsquo;d like to learn more about git, I suggest starting with <a href="https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain">Chapter
10</a> of <a href="https://git-scm.com/book/en/v2">Pro Git</a> (it&rsquo;s free!), then reading chapters 2, 3,
and 7. The rest is optional. In this article, we&rsquo;re going to discuss how you can
apply the tools discussed in the book to a disciplined and productive git
workflow.</p>
<h3 id="the-basics-writing-good-commit-messages">The basics: Writing good commit messages</h3>
<p><a href="https://xkcd.com/1296/"><img src="https://imgs.xkcd.com/comics/git_commit.png" alt=""></a></p>
<p>You may have heard this speech before, but bear with me. Generally, you should
not use <code>git commit -m &quot;Your message here&quot;</code>. Start by configuring git to use
your favorite editor: <code>git config --global core.editor vim</code>, then simply run
<code>git commit</code> alone. Your editor will open and you can fill in the file with your
commit message. The first line should be limited to 50 characters in length, and
should complete this sentence: when applied, this commit will&hellip; &ldquo;Fix text
rendering in CJK languages&rdquo;. &ldquo;Add support for protocol v3&rdquo;. &ldquo;Refactor CRTC
handling&rdquo;. Then, add a single empty line, and expand on this in the <em>extended
commit description</em>, which should be hard-wrapped at 72 columns, and include
details like rationale for the change, tradeoffs and limitations of the
approach, etc.</p>
<p>We use 72 characters because that&rsquo;s <a href="https://tools.ietf.org/html/rfc2822#section-2.1.1">the standard width of an email</a>, and
email is an important tool for git. The 50 character limit is used because the
first line becomes the subject line of your email - and lots of text like
&ldquo;<code>[PATCH linux-usb v2 0/13]</code>&rdquo; can get added to beginning. You might find
wrapping your lines like this annoying and burdensome - but consider that when
working with others, they may not be reading the commit log in the same context
as you. I have a vertical monitor that I often read commit logs on, which is not
going to cram as much text into one line as your 4K 16:9 display could.</p>
<h3 id="each-commit-should-be-a-self-contained-change">Each commit should be a self-contained change</h3>
<p>Every commit should only contain one change - avoid sneaking in little unrelated
changes into the same commit<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. Additionally, avoid breaking one change into
several commits, unless you can refactor the idea into discrete steps - each of
which represents a complete change in its own right. If you have several changes
in your working tree and only need to commit some of them, try <code>git add -i</code> or
<code>git add -p</code>. Additionally, every commit should compile and run all tests
successfully, and should avoid having any known bugs which will be fixed up in a
future commit.</p>
<p>If this is true of your repository, then you can check out any commit and expect
the code to work correctly. This also becomes useful later, for example when
cherry-picking commits into a release branch. Using this approach also allows
<a href="https://git-scm.com/docs/git-bisect">git-bisect</a> to become more useful<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>,
because if you can expect the code to compile and complete tests successfully
for every commit, you can pass <code>git-bisect</code> a script which programmatically
tests a tree for the presence of a bug and avoid false positives. These
self-contained commits with good commit messages can also make it really easy to
prepare release notes with <a href="https://git-scm.com/docs/git-shortlog">git-shortlog</a>,
<a href="https://lkml.org/lkml/2019/1/6/178">like Linus does with Linux releases</a>.</p>
<h3 id="get-it-right-on-the-first-try">Get it right on the first try</h3>
<p>We now come to one of the most important features of git which distinguishes it
from its predecessors: history editing. All version control systems come with a
time machine of some sort, but before git they were mostly read-only. However,
git&rsquo;s time machine is different: you can change the past. In fact, you&rsquo;re
encouraged to! But a word of warning: only change a future which has yet to be
merged into a stable public branch.</p>
<p>The advice in this article - bug-free, self-contained commits with a good commit
message - is hard to get right on the first try. Editing your history, however,
is easy and part of an effective git workflow. Familiarize yourself with
<a href="https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History">git-rebase</a> and use
it liberally. You can use rebase to reorder, combine, delete, edit, and split
commits. One workflow I find myself commonly using is to make some changes to a
file, commit a &ldquo;fixup&rdquo; commit (<code>git commit -m fixup</code>), then use <code>git rebase -i</code>
to squash it into an earlier commit.</p>
<h3 id="other-miscellaneous-tips">Other miscellaneous tips</h3>
<ul>
<li>Read the man pages! Pick a random git man page and read it now. Also, if you
haven&rsquo;t read the top-level git man page (simply <code>man git</code>), do so.</li>
<li>At the bottom of each man page for a high-level git command is usually a list
of low-level git commands that the high-level command relies on. If you want
to learn more about how a high-level git command works, try reading these man
pages, too.</li>
<li>Learn how to specify the commit you want with <a href="https://git-scm.com/book/en/v2/Git-Tools-Revision-Selection">rev selection</a></li>
<li>Branches are useful, but you should learn how to work without them as well to
have a nice set of tools in your belt. Use tools like <code>git pull --rebase</code>,
<code>git send-email -1 HEAD~2</code>, and <code>git push origin HEAD~2:master</code>.</li>
</ul>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>I could stand to take my own advice more often in this respect. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>In a nutshell, git bisect is a tool which does a binary search between two commits in your history, checking out the commits in between one at a time to allow you to test for the presence of a bug. In this manner you can narrow down the commit which introduced a problem. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/03/04/sourcehut-design.html
title: Sourcehut's spartan approach to web design
url: https://drewdevault.com/2019/03/04/sourcehut-design.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p><a href="https://sourcehut.org">Sourcehut</a> is known for its brutalist design, with its
mostly shades-of-gray appearance, conservative color usage, and minimal
distractions throughout. This article aims to share some insights into the
philosophy that guides this design, both for the curious reader and for the
new contributor to the open-source project.</p>
<p>The most important principle is that sr.ht is an engineering tool first and
foremost, and when you&rsquo;re there it&rsquo;s probably because you&rsquo;re in engineering
mode. Therefore, it&rsquo;s important to bring the information you&rsquo;re there for to the
forefront, and minimize distractions. In practice, this means that the first
thing on any page to grab your attention should be the thing that brought you
there. Consider <a href="https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht/service.py">the source file view on git.sr.ht</a>. For
reference, here are similar pages on <a href="https://github.com/torvalds/linux/blob/master/init/main.c">GitHub</a> and
<a href="https://gitlab.freedesktop.org/libinput/libinput/blob/master/src/evdev.c">Gitlab</a>.</p>
<p><a href="https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht/service.py"><img src="https://sr.ht/kkJm.png" alt="Screenshot of git.sr.ht"></a></p>
<style>
img {
  box-shadow: 0 0 2px 2px #888;
  max-width: 90%;
}
</style>
<p>The vast majority of the git.sr.ht page is dedicated to the source code we&rsquo;re
reading here, and it&rsquo;s also where most of the colors are. Your eye is drawn
straight to the content. Any additional information we show on this page is
directly relevant to the content: breadcrumbs to other parts of the tree, file
mode &amp; size, links to other views on this repository. The nav can take you away
from this page, but it&rsquo;s colored a light grey to avoid being distracting and
each link is another engineering tool - no marketing material or fluff. Contrast
with GitHub: a large, dark, attention grabbing navbar with links to direct you
away from the content and towards marketing pages. If you&rsquo;re logged out, you get
a giant sign-up box which pushes the content halfway off the page. Colors here
are also distracting: note the large line of colorful avatars that catches your
eye despite almost certainly being unrelated to your purpose on this page.</p>
<p><img src="https://sr.ht/1qdZ.png" alt="Screenshot of builds.sr.ht"></p>
<p>Colors are used much more conservatively on sourcehut. If you log into
builds.sr.ht and visit the index page, you&rsquo;re greeted with a large blue &ldquo;submit
manifest&rdquo; button, and very little color besides. This is probably why you were
here - so it&rsquo;s made obvious and colorful so your eyes can quickly find it and
get on with your work. Other pages are similar: the todo.sr.ht tracker page has
a large form with a blue &ldquo;submit&rdquo; button for creating a new ticket, email views
on lists.sr.ht have a large blue &ldquo;reply to thread&rdquo; button, and
<a href="https://man.sr.ht">man.sr.ht</a> has a large green button enticing new users
towards the tutorials. Red is also used throughout for dangerous actions, like
deleting things.  Each button also is unambiguous and relies on the text within
itself rather than the text nearby: the git.sr.ht repository deletion page uses
&ldquo;Delete $reponame&rdquo;, rather than &ldquo;Continue&rdquo;.</p>
<p><img src="https://sr.ht/d6Vx.png" alt="Screenshot of repo deletion UI"></p>
<p>The last important point in sourcehut&rsquo;s design is the use of icons, or rather
the lack thereof. Icons are used extremely conservatively on sr.ht. Interactive
icons (things you are expected to click) are never shown without text that
clarifies what happens when you click them. Informational icons usually have a
tooltip which explains their meaning, and are quite rare - only used in cases
where real estate limits the use of text. Assigning an icon to every action or
detail is not necessary and would add more distractions to the screen.</p>
<p>I&rsquo;m not a particularly skilled UI designer, so keeping it simple like this also
helps to make a reasonably nice UI attainable for an engineer-oriented developer
like me. Adding new pages is generally easy and requires little thought by
applying these basic principles throughout, and the simple design doesn&rsquo;t take
long to execute on. It&rsquo;s not perfect, but I like it and I&rsquo;ve received positive
feedback from my users.</p>


guid: https://drewdevault.com/2019/03/11/Sway-1.0-released.html
title: Announcing the release of sway 1.0
url: https://drewdevault.com/2019/03/11/Sway-1.0-released.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>1,315 days after I started the <a href="https://swaywm.org">sway</a> project, it&rsquo;s finally
time for <a href="https://github.com/swaywm/sway/releases/tag/1.0">sway 1.0</a>! I had no
idea at the time how much work I was in for, or how many talented people would
join and support the project with me. In order to complete this project, we have
had to rewrite the entire Linux desktop nearly from scratch. Nearly 300 people
worked together, together writing over 9,000 commits and almost 100,000 lines of
code, to bring you this release.</p>
<p><small class="text-muted">Sway is an i3-compatible Wayland desktop for Linux and FreeBSD</small></p>
<p>1.0 is the first stable release of sway and represents a consistent, flexible,
and powerful desktop environment for Linux and FreeBSD. We hope you&rsquo;ll enjoy it!
If the last sway release you used was 0.15 or earlier, you&rsquo;re in for a shock.
0.15 was a buggy, frustrating desktop to use, but sway 1.0 has been completely
overhauled and represents a much more capable desktop. It&rsquo;s almost impossible to
summarize all of the changes which makes 1.0 great. Sway 1.0 adds a huge variety
of features which were sorely missed on 0.x, improves performance in every
respect, offers a more faithful implementation of Wayland, and exists as a
positive political force in the Wayland ecosystem pushing for standardization
and cooperation among Wayland projects.</p>
<p>When planning the future of sway, we realized that the Wayland ecosystem was
sorely in need of a stable &amp; flexible common base library to encapsulate all of
the difficult and complex facets of building a desktop. To this end, I decided
we would build <a href="https://github.com/swaywm/wlroots">wlroots</a>. It&rsquo;s been a
smashing success. This project has become very important to the Linux desktop
ecosystem, and the benefits we reap from it have been shared with the community
at large. <a href="https://github.com/swaywm/wlroots/wiki/Projects-which-use-wlroots">Dozens of projects</a> are using it today, and soon you&rsquo;ll
find it underneath most Linux desktops, on your phone, in your VR environment,
and more. Its influence extends beyond its borders as well, as we develop and
push for standards throughout Wayland.</p>
<p>Through this work we have also helped to build a broader ecosystem of tools
built on interoperable standards which you may find useful in your new sway 1.0
desktop. Here are a few of my favorites - each of which is compatible with many
Wayland compositors:</p>
<ul>
<li><a href="https://github.com/swaywm/swayidle">swayidle</a>: idle management daemon</li>
<li><a href="https://github.com/swaywm/swaylock">swaylock</a>: lock screen</li>
<li><a href="https://github.com/emersion/mako">mako</a>: notification daemon</li>
<li><a href="https://github.com/emersion/grim">grim</a>: screenshot tool</li>
<li><a href="https://github.com/emersion/slurp">slurp</a>: interactive region selection</li>
<li><a href="https://github.com/ammen99/wf-recorder">wf-recorder</a>: video capture tool</li>
<li><a href="https://github.com/Alexays/Waybar">waybar</a>: alternative panel</li>
<li><a href="https://source.puri.sm/Librem5/virtboard">virtboard</a>: on-screen keyboard</li>
<li><a href="https://github.com/bugaevc/wl-clipboard">wl-clipboard</a>: xclip replacement</li>
<li><a href="https://github.com/xyproto/wallutils">wallutils</a>: fancy wallpaper manager</li>
</ul>
<hr>
<p>None of this would be possible without the support of sway&rsquo;s and wlroots'
talented contributors. Hundreds of people worked together on this. I&rsquo;d like to
give special thanks to our core contributors: Brian Ashworth, Ian Fan, Ryan
Dwyer, Scott Anderson, and Simon Ser. Thanks are also in order for those who
have helped wlroots fit into the broader ecosystem - thanks to Purism for their
help on wlroots, KDE &amp; Canonical for their help on protocol standardization. I
also owe thanks to all of the other projects which use wlroots, particularly
including Way Cooler, Wayfire, and Waymonad, who all have made substantial
contributions to wlroots in their pursit of the best Wayland desktop.</p>
<p>I&rsquo;d also of course like to thank all of the users who have donated to support
my work, which I now do full-time, which has had and I hope will continue to
have a positive impact on the project and those around it. Please consider
<a href="https://drewdevault.com/donate">donating</a> to support the future of sway &amp;
wlroots if you haven&rsquo;t yet.</p>
<p>Though sway today is already stable and powerful, we&rsquo;re not done yet. We plan to
continue improving performance &amp; stability, adding useful desktop features,
taking advantage of better hardware, and bringing sway to more users. Here&rsquo;s
some of what we have planned for future releases:</p>
<ul>
<li>Better Wayland-native tools for internationalized input methods like CJK</li>
<li>Better accessibility tools including improved screen reader support,
high-contrast mode, a magnifying glass tool, and so on</li>
<li>Integration with xdg-portal &amp; pipewire for interoperable screen capture</li>
<li>Improved touch screen support for use on the <a href="https://puri.sm/products/librem-5/">Librem
5</a> and on
<a href="https://postmarketos.org/">postmarketOS</a></li>
<li>Better support for drawing tablets and additional hardware</li>
<li>Sandboxing and security features</li>
</ul>
<p>As with all sway features, we intend to have the best-in-class implementations
of these features and set the bar as high as we can for everyone else. We&rsquo;re
looking forward to your continued support!</p>


guid: https://drewdevault.com/2019/03/15/Status-update-March-2019.html
title: Status update, March 2019
url: https://drewdevault.com/2019/03/15/Status-update-March-2019.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>My todo list is getting completed at a pace it&rsquo;s never seen before, and growing
at a new pace, too. This full-time FOSS gig is really killing it! As the good
weather finally starts to roll in, it&rsquo;s time for March&rsquo;s status update. Note: I
posted updates <a href="https://www.patreon.com/sircmpwn/posts">on Patreon</a> before, but will start posting here
instead. This medium doesn&rsquo;t depend on a proprietary service, allows for richer
content, and is useful for my supporters who support my work via other donation
platforms.</p>
<p>Sway 1.0 has been released! <img style="display: inline; height: 1.2rem"
src="https://drewdevault.com/img/party.png" /> I wrote a <a href="https://drewdevault.com/2019/03/11/Sway-1.0-released.html">detailed
write-up</a> on the release and our future
plans separately, which I encourage you to read if you haven&rsquo;t already. However,
I do have some additional progress to share outside of the big sway 1.0 news.
In the last update, I mentioned that I got a Librem 5 devkit from Guido
Günther<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> at FOSDEM. My plans were to get this up and running with sway and
start improving touch support, and I&rsquo;ve accomplished both:</p>
<p><img src="https://sr.ht/fGxf.jpg" alt="A picture of a Librem5 devkit running pmOS and sway"></p>
<p>As you can see, I also got <a href="https://postmarketos.org/">postmarketOS</a> running,
and I love it - I hope to work with them a lot in the future. The <a href="https://github.com/swaywm/sway/pull/3711">first
patch</a> for improving touch support in
sway has landed and I&rsquo;ll be writing more in the future. I also sent some patches
to Purism&rsquo;s <a href="https://source.puri.sm/Librem5/virtboard">virtboard</a> project, an
on-screen keyboard, making it more useful for Sway users. I hope to make an OSK
of my own at some point, with multiple layouts, CJK support, and client-aware
autocompletion, in the future. Until then, an improved virtboard is a nice
stop-gap :) I&rsquo;ve also been working on wlroots a bit, including <a href="https://github.com/swaywm/wlroots/pull/1578">a patch adding
remote desktop support</a>.</p>
<p>In other Wayland news, I&rsquo;ve also taken a part time contract to build a module
integrating wlroots with the <a href="https://godotengine.org/">Godot game engine</a>:
<a href="https://git.sr.ht/~sircmpwn/gdwlroots">gdwlroots</a>. The long-term goal is to
build a VR compositor based on Godot and develop standards for Wayland
applications to have 3D content. 100% of this work is free software (MIT
licensed) and will bring improvements to both the wlroots and Godot ecosystems.
Next week I&rsquo;ll be starting work on adding a Wayland backend to Godot so that
Godot-based games can run on Wayland compositors directly. Here&rsquo;s an example
compositor running on Godot:</p>
<p>&lt;video
src=&ldquo;<a href="https://sr.ht/9bV-.webm%22">https://sr.ht/9bV-.webm&quot;</a>
autoplay muted loop controls
style=&ldquo;max-width: 100%;&rdquo;</p>
<blockquote>
</video>
</blockquote>
<p>I&rsquo;ve also made some significant progress on
<a href="https://git.sr.ht/~sircmpwn/aerc2">aerc2</a>. I have fleshed out the command
subsystem, rigged up keybindings, and implemented the message list, and along
with it all of the asynchronous communication between the UI thread, network
thread, and mail server. I think at this point most of the unknowns are solved
with aerc2, and the rest just remains to be implemented. I&rsquo;m glad I chose to
rewrite it from C, though my love for C still runs deep. The Go ecosystem is
much better suited to the complex problems and dependency tree of software like
aerc, plus has a nice concurrency model for aerc&rsquo;s async design.<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> The next
major problem to address is the embedded terminal emulator, which I hope to
start working on soon.</p>
<script
  id="asciicast-pafXXANiWHY9MOH2yXdVHHJRd"
  src="https://asciinema.org/a/pafXXANiWHY9MOH2yXdVHHJRd.js" async
></script>
<p>aerc2&rsquo;s progress is a great example of my marginalized projects
becoming my side projects, as my side projects become my full-time job, and thus
all of them are developing at a substantially improved pace. The productivity
increase is pretty crazy. I&rsquo;m really thankful to everyone who&rsquo;s supporting my
work, and excited to keep building crazy cool software thanks to you.</p>
<p>I was meaning to work on RISC-V this month, but I&rsquo;ve been a little bit
distracted by everything else. However, there has been some discussion about how
to approach upstreaming and I&rsquo;m planning on tackling this next week. I also
spent some time putting together a custom 1U I can install in my datacenter for
a more permanent RISC-V setup. Some of this is working towards getting RISC-V
ready for builds.sr.ht users to take advantage of - that relay is for cutting
power to the board to force a reboot when it misbehaves - but a lot of this is
also useful for my own purposes in porting musl &amp; Alpine Linux.</p>
<p><img src="https://sr.ht/M7me.jpg" alt="Picture of a 1U chassis with a bunch of custom components within"></p>
<p>One problem I&rsquo;m still trying to solve is the microSD card. I don&rsquo;t want to run
untrusted builds.sr.ht code when that microSD card is plugged in. I&rsquo;ve been
working on some prototyping (breaking out the old soldering iron) to make a
microSD&hellip; thing, which I can plug into this and physically cut VCC to the
microSD card with that relay I have rigged up. This is pretty hard, and my
initial attempts were unsuccessful. If anyone knowledgable about this has ideas,
please get in touch.</p>
<p>Outside of RISC-V, I have been contributing to Alpine Linux a lot more lately in
general. I adopted the sway &amp; wlroots packages, have been working on improved
PyQt support, cleaning up Python packages, clearing out the nonfree MongoDB
packages, and more. I also added a bunch of new packages for miscellaneous
stuff, including alacritty, font-noto-cjk, nethack, and Simon Ser&rsquo;s
<a href="https://github.com/emersion/go-dkim">go-dkim</a> milter. Most importantly,
however, I&rsquo;ve started
<a href="https://wiki.alpinelinux.org/wiki/Python_package_policies">planning</a> and
<a href="https://lists.alpinelinux.org/alpine-devel/6465.html">discussing</a> a Python
overhaul project in aports with the Alpine team, which will including cleaning
up all of the Python patches and starting on Python 2 removal. I depend a lot on
Alpine and its Python support, so I&rsquo;m excited to be working on these
improvements!</p>
<p>I have some Sourcehut news as well. Like usual, there&rsquo;ll be a detailed
Sourcehut-specific update posted to the
<a href="https://lists.sr.ht/~sircmpwn/sr.ht-announce">sr.ht-announce</a> mailing list
later on. With Ludovic Chabant&rsquo;s help, there have been continued improvements to
Mercurial support, notably adding builds.sr.ht integration as of yesterday.
Thanks Ludovic! We&rsquo;ve also been talking to some NetBSD folks who may be
interested in using Sourcehut to host the NetBSD code once they finish their
CVS-&gt;Hg migration, and we&rsquo;ve been improving the performance for large
repositories during their experiments on sr.ht.</p>
<p>There&rsquo;s a bunch more going on with Sourcehut - paste.sr.ht, APIs, a command line
interface for those APIs, webhooks, and more still - check out the email on
sr.ht-announce later. That&rsquo;s all I have for you today. Thank you for your
support, and until next time!</p>
<small class="text-muted">
This work was possible thanks to users who support me financially. Please
consider <a href="https://drewdevault.com/donate">donating to my work</a> or <a
href="https://sourcehut.org">buying a sourcehut.org subscription</a>. Thank you!
</small>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>A Purism employee that works closely with wlroots on the Librem 5 <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>&ldquo;aerc&rdquo; stands for &ldquo;asynchronous email reading client&rdquo;, after all. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/03/25/Rust-is-not-a-good-C-replacement.html
title: Rust is not a good C replacement
url: https://drewdevault.com/2019/03/25/Rust-is-not-a-good-C-replacement.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I have a saying that summarizes my opinion of Rust compared to Go: &ldquo;Go is the
result of C programmers designing a new programming language, and Rust is the
result of C++ programmers designing a new programming language&rdquo;. This isn&rsquo;t just
a metaphor - Go was designed by plan9 alumni, an operating system written in C
and the source of inspiration for many of Go&rsquo;s features, and Rust was designed
by the folks at Mozilla - whose flagship product is one of the largest C++
codebases in the world.</p>
<p>The values of good C++ programmers are incompatible with the values of good C
programmers<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. Rust is a decent C++ replacement if you have the same goals as
C++, but if you don&rsquo;t, the design has very similar drawbacks. Both Rust and C++
are what I like to call &ldquo;kitchen sink&rdquo; programming languages, with the obvious
implication. These languages solve problems by adding more language features. A
language like C solves problems by writing more C code.</p>
<p>I did some back of the napkin estimates of the rate at which these languages
become more complex, based on the rate at which they add features per year. My
approach wasn&rsquo;t very scientific, but I&rsquo;m sure the point comes across.</p>
<ul>
<li><strong>C: 0.73 new features per year</strong>, measured by the number of bullet points in
the C11 article on Wikipedia which summarizes the changes from C99, adjusted to
account for the fact that C18 introduced no new features.</li>
<li><strong>Go: 2 new features per year</strong>, measured by the number of new features listed
on the Wikipedia summary of new Go versions.</li>
<li><strong>C++: 11.3 new features per year</strong>, measured by the number of bullet points
in the C++17 article which summarizes the changes from C++14.</li>
<li><strong>Rust: 15 new features per year</strong>, measured by the number of headers in the
release notes of major Rust versions over the past year, minus things like
linters.</li>
</ul>
<p>This speaks volumes to the stability of these languages, but more importantly it
speaks to their complexity. Over time it rapidly becomes difficult for one to
keep an up-to-date mental map of Rust and how to solve your problems
idiomatically. A Rust program written last year already looks outdated, whereas
a C program written ten years ago has pretty good odds of being just fine.
Systems programmers don&rsquo;t want shiny things - we just want things that work.
That really cool feature $other_language has? Not interested. It&rsquo;ll be more
trouble than it&rsquo;s worth.</p>
<p>With the philosophical wish-wash out of the way and the tone set, let me go over
some more specific problems when considering Rust as a C replacement.</p>
<p><strong>C is the most portable programming language</strong>. Rust actually has a pretty
admirable selection of supported targets for a new language (thanks mostly to
LLVM), but it pales in comparison to C, which runs on almost <em>everything</em>. A new
CPU architecture or operating system can barely be considered to exist until it
has a C compiler. And once it does, it unlocks access to a vast repository of
software written in C. Many other programming languages, such as Ruby and
Python, are implemented in C and you get those for free too.</p>
<p><strong>C has a spec</strong>. No spec means there&rsquo;s nothing keeping rustc honest. Any
behavior it exhibits could change tomorrow. Some weird thing it does could be a
feature <em>or</em> a bug. There&rsquo;s no way to know until your code breaks. That they
can&rsquo;t slow down to pin down exactly what defines Rust is also indicative of an
immature language.</p>
<iframe
  src="https://cmpwn.com/@sir/100437209244243864/embed"
  class="mastodon-embed"
  style="max-width: 100%; border: 0; margin: 0 auto; display: block;"
  width="400"></iframe>
<script src="https://cmpwn.com/embed.js" async="async"></script>
<p><strong>C has many implementations</strong>. C has many competing compilers. They all work
together stressing out the spec, fishing out the loosely defined corners, and
pinning down exactly what C is. Code that compiles in one and not another is
indicative of a bug in one of them, which gives a nice extra layer of testing to
each. By having many implementations, we force C to be well defined, and this is
good for the language and its long-term stability. Rustc could stand to have
some competition as well, maybe it would get faster<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup></p>
<p><strong>C has a consistent &amp; stable ABI</strong>. The System-V ABI is supported on a wide
variety of systems and has been mostly agreed upon by now. Rust, on the other
hand, has no stable internal ABI. You have to compile and link everything all in
one go on the same version of the Rust compiler. The only code which can
interact with the rest of the ecosystem is unidiomatic Rust, written at some
kind of checkpoint between Rust and the outside world. The outside world exists,
it speaks System-V, and us systems programmers spend a lot of our time talking
to it.</p>
<p><strong>Cargo is mandatory</strong>. On a similar line of thought, Rust&rsquo;s compiler flags are
not stable. Attempts to integrate it with other build systems have been met with
hostility from the Rust &amp; Cargo teams. The outside world exists, and us systems
programmers spend a lot of our time integrating things. Rust refuses to play
along.</p>
<p><strong>Concurrency is generally a bad thing.</strong> Serial programs have X problems, and
parallel programs have X<sup>Y</sup> problems, where Y is the amount of
parallelism you introduce. Parallelism in C is a pain in the ass for sure, and
this is one reason I find Go much more suitable to those cases. However, nearly
all programs needn&rsquo;t be parallel. A program which uses poll effectively is going
to be simpler, reasonably performant, and have orders of magnitude fewer bugs.
&ldquo;Fearless concurrency&rdquo; allows you to fearlessly employ bad software design 9
times out of 10.</p>
<p><strong>Safety</strong>. Yes, Rust is more safe. I don&rsquo;t really care. In light of all of
these problems, I&rsquo;ll take my segfaults and buffer overflows. I especially refuse
to &ldquo;rewrite it in Rust&rdquo; - because no matter what, rewriting an entire program
from scratch is <em>always</em> going to introduce more bugs than maintaining the C
program ever would. I don&rsquo;t care what language you rewrite it in.</p>
<hr>
<p>C is far from the perfect language - it has many flaws. However, its
replacement will be simpler - not more complex. Consider Go, which has had a lot
of success in supplanting C for many problems. It does this by specializing on
certain classes of programs and addressing them with the simplest solution
possible. It hasn&rsquo;t completely replaced C, but it has made a substantial dent in
its problem space - more than I can really say for Rust (which has made similar
strides for C++, but definitely not for C).</p>
<p>The kitchen sink approach doesn&rsquo;t work. Rust will eventually fail to the &ldquo;jack
of all trades, master of none&rdquo; problem that C++ has. Wise languages designers
start small and stay small. Wise systems programmers extend this philosophy to
designing entire systems, and Rust is probably not going to be invited. I
understand that many people, particularly those already enamored with Rust,
won&rsquo;t agree with much of this article. But now you know why we are still writing
C, and hopefully you&rsquo;ll stop bloody bothering us about it.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Aside: the term &ldquo;C/C++&rdquo; infuriates me. They are completely different languages. Idiomatic C++ looks nothing like idiomatic C. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Rust does have one competing compiler, but without a spec it&rsquo;s hard to define its level of compatibility and correctness, and it&rsquo;s always playing catch-up. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/04/02/NewPipe-represents-the-best-of-FOSS.html
title: NewPipe represents the best of FOSS
url: https://drewdevault.com/2019/04/02/NewPipe-represents-the-best-of-FOSS.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p><a href="https://newpipe.schabi.org/">NewPipe</a> is a free and open-source Android
application for browsing &amp; watching YouTube. In my opinion, NewPipe is a perfect
case-study in why free &amp; open source software is great and how our values differ
from proprietary software in important ways. There&rsquo;s one simple reason: it&rsquo;s
better than the proprietary YouTube app, in every conceivable way, for free.</p>
<p>NewPipe is better because it&rsquo;s user-centric software. It exists to make its
users lives better, not to enrich its overseers. Because of this, NewPipe has
many features which are deliberately omitted from the proprietary app, such as:</p>
<ul>
<li>No advertisements<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></li>
<li>Playing any video in the background</li>
<li>Downloading videos (or their audio tracks alone) to play offline</li>
<li>Playing videos in a pop-up player</li>
<li>Subscribing to channels without a YouTube account</li>
<li>Importing and exporting subscriptions</li>
<li>Showing subscriptions in chronological order</li>
<li>It supports streaming services other than YouTube<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup></li>
</ul>
<p>YouTube supports some of these&hellip; for $12/month. Isn&rsquo;t that a bit excessive?
Other features it doesn&rsquo;t support at all. On top of that, YouTube is constantly
gathering data about you and making decisions which put their interests ahead of
yours, whereas NewPipe never phones home and consistently adds new features that
put users first. The proprietary app is <em>exploitative</em> of users, and NewPipe is
<em>empowering</em> users.</p>
<p>There are a lot of political and philosophical reasons to use &amp; support free and
open source software. Sometimes it&rsquo;s hard to get people on board with FOSS by
pitching them these first. NewPipe is a great model because it&rsquo;s straight up
<em>better</em>, and better for reasons that make these philosophical points obvious
and poignant. The NewPipe project was started by <a href="https://schabi.org/">Christian
Schabesberger</a>, is co-maintained by a <a href="https://github.com/orgs/TeamNewPipe/people">team of 6</a>, and
has been contributed to by over <a href="https://github.com/TeamNewPipe/NewPipe/graphs/contributors">300 people</a>. You can donate
<a href="https://newpipe.schabi.org/donate/">here</a>.  NewPipe represents the best of our community. Thanks!
<img src="https://drewdevault.com/img/heart.png" style="display: inline; width: 1.2rem; top: 0.2rem; position: relative" /></p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Support your content creators with tools like Liberapay and Patreon! <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>At least in theory&hellip; basic SoundCloud support is working and more services are coming soon. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/04/15/Status-update-April-2019.html
title: Status update, April 2019
url: https://drewdevault.com/2019/04/15/Status-update-April-2019.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Spring is here, and I&rsquo;m already miserable in the heat. Crazy weather here in
Philadelphia - I was woken up at 3 AM by my phone buzzing, telling me to take
immediate shelter from a tornado. But with my A/C cranked up and the tornado
safely passed, I&rsquo;ve been able to get a lot of work done.</p>
<p>The project with the most impressive progress is
<a href="https://git.sr.ht/~sircmpwn/aerc2">aerc2</a>. It can now read emails, including
filtering them through arbitrary commands for highlighting diffs or coloring
quotes, or even rendering HTML email with a TUI browser like w3m.</p>
<script
  id="asciicast-vy5GmO0tBjppr4G2LSQONIFjH"
  src="https://asciinema.org/a/pafXXANiWHY9MOH2yXdVHHJRd.js" async
></script>
<p>Here&rsquo;s another demo focusing on the embedded terminal emulator which makes this
possible:</p>
<script
  id="asciicast-N57RaPJqwQD2h0AejLGDWrSi9"
  src="https://asciinema.org/a/pafXXANiWHY9MOH2yXdVHHJRd.js" async
></script>
<p>Keybindings are also working, which are configured simiarly to vim - each
keybinding simulates a series of keystrokes, which all eventually boil down to
an ex-style command. I&rsquo;ve bought a domain for aerc, and I&rsquo;ll be populating it
with some marketing content and a nice tour of the features soon. I hope to have
time to work on sending emails this month as well. In the immediate future, I
need to fix some crashiness that occurs in some situations.</p>
<p>In other email-related news, <a href="https://git-send-email.io">git-send-email.io</a> is
now live, an interactive tutorial on using email with git. This workflow is the
one sourcehut focuses on, and is also used by a large number of important free
software projects, like Linux, gcc, clang, glibc, musl, ffmpeg, vim, emacs,
coreutils&hellip; and many, many more. Check it out!</p>
<p>I also spent a fair bit of time working on lists.sr.ht this month. Alpine Linux
has provisioned some infrastructure for a likely migration from their current
mailing list solution (mlmmj+hypermail) to one based on lists.sr.ht, which I
deployed a lists.sr.ht instance to for them, and trained them on some
administrative aspects of lists.sr.ht. User-facing improvments that came from
this work include tools for importing and exporting mail spools from lists,
better access controls, moderation tools, and per-list mime whitelisting and
blacklisting. Admin-facing tools include support for a wider variety of MTA
configurations and redirects to continue supporting old incoming mail addresses
when migrating from another mailing list system.</p>
<p>Stepping outside the realm of email, let&rsquo;s talk about Wayland. Since Sway 1.0,
development has continued at a modest pace, fixing a variety of small bugs and
further improving i3 compatibility. We&rsquo;re getting ready to split swaybg into a
standalone project which can be used on other Wayland compositors soon, too. I
also have been working more on Godot, and have switched gears towards adding a
Wayland backend to Godot upstream - so you can play Godot-based video games on
Wayland. I&rsquo;m still working with upstream and some other interested contributors
on the best way to integrate these changes upstream, but I more or less
completed a working port with support for nearly all of Godot&rsquo;s platform
abstractions.</p>
<p><a href="https://sr.ht/fOvB.png"><img src="https://sr.ht/fOvB.png" alt="Godot editor running on Wayland with HiDPI support"></a></p>
<p>In smaller project news, I spent an afternoon putting together a home-grown
video livestreaming platform a few weeks ago. The result:
<a href="https://live.drewdevault.com">live.drewdevault.com</a>. Once upon a time I was
livestreaming programming sessions on Twitch.tv, and in the future I&rsquo;d like to
do this more often on my new platform. This one is open source and built on the
shoulders of free software tools. I announce new streams on
<a href="https://cmpwn.com/@sir">Mastodon</a>, join us for the next one!</p>
<p>I&rsquo;m also starting on another project called cozy, which is yak-shaving for
several other projects I have in mind. It&rsquo;s kind of ambitious&hellip; it&rsquo;s a full
end-to-end C compiler toolchain. One of my goals (which, when completed, can
unblock other tasks before cozy as a whole is done) is to make the parser work
as a standalone library for reading, writing, and maniuplating the C AST. I&rsquo;ve
completed the lexer and basic yacc grammar, and I&rsquo;m working on extracting an AST
from the parser. I only started this weekend, so it&rsquo;s pretty early on.</p>
<p>I&rsquo;ll leave you with a fun weekend project I did shortly after the last update:
<a href="https://qlock.drewdevault.com/">otaqlock</a>. The server this runs on isn&rsquo;t awash
with bandwidth and the site doesn&rsquo;t work great on mobile - so your milage may
vary - but it is a cool artsy restoration project nonetheless. Until next time,
and thank you for your support!</p>
<small class="text-muted">
This work was possible thanks to users who support me financially. Please
consider <a href="https://drewdevault.com/donate">donating to my work</a> or <a
href="https://sourcehut.org">buying a sourcehut.org subscription</a>. Thank you!
</small>


guid: https://drewdevault.com/2019/04/15/Announcing-first-class-hg-support-on-sourcehut.html
title: Announcing first-class Mercurial support on Sourcehut
url: https://drewdevault.com/2019/04/15/Announcing-first-class-hg-support-on-sourcehut.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>I&rsquo;m pleased to announce that the final pieces have fallen into place for
<a href="https://www.mercurial-scm.org/">Mercurial</a> support on
<a href="https://sourcehut.org">SourceHut</a>, which is now on-par with our git offering.
Special thanks are owed to SourceHut contributor Ludovic Chabant, who has been
instrumental in adding Mercurial support to SourceHut. You may have heard about
it while this was still experimental - but I&rsquo;m happy to tell you that we have
now completely integrated Mercurial support into SourceHut! Want to try it out?
Check out <a href="https://man.sr.ht/tutorials/set-up-account-and-hg.md">the tutorial</a>.</p>
<p>Mercurial support on SourceHut includes all of the trimmings, including CI
support via <a href="https://builds.sr.ht">builds.sr.ht</a> and email-driven collaboration
on <a href="https://lists.sr.ht">lists.sr.ht</a>. Of course, it&rsquo;s also 100%
free-as-in-freedom, open source software (<a href="https://hg.sr.ht/~sircmpwn/hg.sr.ht">hosted on
itself</a>) that you can <a href="https://man.sr.ht/hg.sr.ht/installation.md">deploy on your own
servers</a>. We&rsquo;ve tested hg.sr.ht
on some of the largest Mercurial repositories out there, including
mozilla-central and NetBSD src. The NetBSD project in particular has been very
helpful, walking us through their CVS to Hg conversion and stress-testing
hg.sr.ht with the resulting giant repositories. I&rsquo;m looking forward to working
more with them in the future!</p>
<p>The Mercurial community is actively innovating their software, and we&rsquo;ll be
right behind them. I&rsquo;m excited to provide a platform for elevating the Mercurial
community. There weren&rsquo;t a lot of good options for Mercurial fans before
SourceHut. Let&rsquo;s fix that together! SourceHut will be taking a more active role
in the Hg community, just like we have for git, and together we&rsquo;ll build a great
platform for software development.</p>
<p>I&rsquo;ll see you in Paris in May, at the <a href="https://www.mercurial-scm.org/pipermail/mercurial/2019-April/051196.html">inaugural Mercurial
conference</a>!</p>
<hr>
<p>Hg support on SourceHut was largely written by members of the Mercurial
community. If there are other version control communities interested in
SourceHut support, please <a href="mailto:~sircmpwn/sr.ht-dev@lists.sr.ht">reach out</a>!</p>


guid: https://drewdevault.com/2019/04/19/Your-VPN-is-a-serious-choice.html
title: Choosing a VPN service is a serious decision
url: https://drewdevault.com/2019/04/19/Your-VPN-is-a-serious-choice.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>There&rsquo;s a disturbing trend in the past year or so of various VPN companies
advertising to the general, non-technical public. It&rsquo;s great that the general
public is starting to become more aware of their privacy online, but I&rsquo;m not a
fan of these companies exploiting public paranoia to peddle their wares. Using
a VPN in the first place has potentially grave consequences for your privacy -
and can often be worse than not using one in the first place.</p>
<p>It&rsquo;s true that, generally speaking, when you use a VPN, the websites you visit
don&rsquo;t have access to your original IP address, which can be used to derive your
approximate location (often not more specific than your city or neighborhood).
But that&rsquo;s not true of the VPN provider themselves - who can identify you much
more precisely because you used your VPN login to access the service.
Additionally, they can promise not to siphon off your data and write it down
somewhere - tracking you, selling it to advertisers, handing it over to law
enforcement - but they <em>could</em> and you&rsquo;d be none the wiser. By routing all of
your traffic through a VPN, <em>you route all of your traffic through a VPN</em>.</p>
<p>Another advantage offered by VPNs is that they can prevent your ISP from knowing
what you&rsquo;re doing online. If you don&rsquo;t trust your ISP but you do trust your VPN,
this makes a lot of sense. It also makes sense if you&rsquo;re on an unfamiliar
network, like airport WiFi. However, it&rsquo;s still quite important that you <em>do</em>
trust the VPN on the other end. You need to do research. What country are they
based in, and what&rsquo;s their diplomatic relationship with your home country? What
kind of power the local authorities have to force them to record &amp; disclose your
traffic? Are they backed by venture capitalists who expect infinite growth, and
will they eventually have to meet those demands by way of selling your
information to advertisers? What happens to you when their business is going
poorly? How much do you trust their security competency - are they likely to be
hacked? If you haven&rsquo;t answered all of these questions yourself, then you should
not use a VPN.</p>
<p>Even more alarming than the large advertising campaigns which have been popular
in the past few months is push-button VPN services which are coming
pre-installed on consumer hardware and software. These bother me because they&rsquo;re
implemented by programmers who should understand this stuff and know better than
to write the code. Opera now has a push-button VPN pre-bundled which is free and
tells you little about the service before happily sending all of your traffic
through it.  Do you trust a Chinese web browser&rsquo;s free VPN to behave in your
best interests?  Purism also recently announced a collaboration with Private
Internet Access to ship a VPN in their upcoming Librem 5. I consider this highly
irresponsible of Purism, and actually discussed the matter at some length with
Todd Weaver (the CEO) over email. We need to stop making it easy for users to
siphon all of their data into the hands of someone they don&rsquo;t know.</p>
<p>For anyone who needs a VPN but isn&rsquo;t comfortable using one of these companies,
there are other choices. First, consider that any website you visit with HTTPs
support (identified by the little green lock in the address bar on your web
browser) is already encrypting all of your traffic so it cannot be read or
tampered with. This discloses your IP address to the operator of that website
and discloses that you visited that website to your ISP, but does not disclose
any data you sent to them, or any content they sent to you, to your ISP or any
eavesdroppers. If you&rsquo;re careful to use HTTPS (and other forms of SSL for
things like email), that can often be enough.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p>
<p>If that&rsquo;s not enough, the ironclad solution is
<a href="https://www.torproject.org/">Tor</a>. When you connect to a website on Tor, it (1)
hides your IP address from the website and any eavesdroppers, (2) hides who
you&rsquo;re talking to from your ISP, and (3) hides what you&rsquo;re talking about from
the ISP. In some cases (onion services), it even hides the origin of the service
you&rsquo;re talking to from <em>you</em>. Tor comes with its own set of limitations and
pitfalls for privacy &amp; security, which you should <a href="https://2019.www.torproject.org/download/download.html.en#Warning">read about and
understand</a>
before using it. Bad actors on the Tor network can read and tamper with your
traffic if you aren&rsquo;t using SSL or Onion routing.</p>
<p>Finally, if you have some technical know-how, you can set up your own VPN. If
you have a server somewhere (or rent one from a VPS provider), you can install a
VPN on it. I suggest <a href="https://www.wireguard.com/">Wireguard</a> (easiest, Linux
only) or <a href="https://openvpn.net">OpenVPN</a> (more difficult, works on everything).
Once again, this comes with its own limitations. You&rsquo;ll always be using a
consistent IP address that services you visit can remember to track you, and you
get a new ISP (whoever your VPS provider uses). This&rsquo;ll generally route you
through commercial ISPs, though, who are much less likely to do obnoxious crap
like injecting ads in webpages or redirecting your failed DNS queries to &ldquo;search
results&rdquo; (i.e. more ads). You&rsquo;ll need to vet your VPS provider and their ISP
with equal care.</p>
<p>Understand who handles your data - encrypted and unencrypted - before you share
it.  No matter your approach, you should also always install an adblocker (I
strongly recommend <a href="https://github.com/gorhill/uBlock/#installation">uBlock
Origin</a>), stick to
HTTPS-enabled websites, and be suspicious of and diligent about every piece of
software, every browser extension, every app you install, and every website you
visit. Most of them are trying to spy on you.</p>
<p>Related articles:</p>
<ul>
<li><a href="https://schub.io/blog/2019/04/08/very-precarious-narrative.html">VPN - a Very Precarious Narrative - Dennis Schubert</a></li>
<li><a href="https://www.skadligkod.se/vpn/the-trustworthy-of-vpn-review-sites-and-how-affiliate-programs-affects-their-opinion/">The trustworthy of VPN review sites and how affiliate programs affects their opinion</a></li>
</ul>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>A reader points out that HTTPS can also be tampered with. If someone else administrates your computer (such as your employer), they can install custom certificates that allow them to tamper with your traffic. This is also sometimes done by software you install on your system, like antivirus software (which more times than not, is a virus itself). Additionally, anyone who can strongarm a certificate authority (state actors) may be able to issue an illegitimate certificate for the same purpose. The only communication method I know of which has no known flaws is onion routing on Tor. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/04/23/Using-cage-for-a-seamless-RDP-Wayland-desktop.html
title: Using Cage for a seamless remote Wayland session
url: https://drewdevault.com/2019/04/23/Using-cage-for-a-seamless-RDP-Wayland-desktop.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>Congratulations to Jente Hidskes on <a href="https://www.hjdskes.nl/blog/cage-01/">the first release of
Cage</a>! Cage is a Wayland compositor
designed for kiosks - though, as you&rsquo;ll shortly find out, is useful in many
unexpected ways. It launches a single application, in fullscreen, and exits the
compositor when that application exits. This lets you basically add a
DRM+KMS+libinput session to any Wayland-compatible application (or X application
via XWayland) and run it in a tiny wlroots compositor.</p>
<p>I actually was planning on writing something like this at some point (for a
project which still hasn&rsquo;t really come off the ground yet), so I was excited
when Jente <a href="https://www.hjdskes.nl/blog/cage/">announced it</a> in December. With
the addition of the <a href="https://github.com/swaywm/wlroots/pull/1578">RDP backend</a>
in wlroots, I thought it would be cool to combine these to make a seamless
remote desktop experience. In short, I installed
<a href="http://www.freerdp.com/">FreeRDP</a> and Cage on my laptop, and
<a href="https://swaywm.org">sway</a> on my desktop. On my desktop, I <a href="https://github.com/swaywm/wlroots/blob/master/docs/env_vars.md#rdp-backend">generated TLS
certificates per the wlroots
docs</a>
and ran sway like so:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh"><span class="nv">WLR_RDP_TLS_CERT_PATH</span><span class="o">=</span><span class="nv">$HOME</span>/tls.crt <span class="se">\
</span><span class="se"></span><span class="nv">WLR_RDP_TLS_KEY_PATH</span><span class="o">=</span><span class="nv">$HOME</span>/tls.key <span class="se">\
</span><span class="se"></span><span class="nv">WLR_BACKENDS</span><span class="o">=</span>rdp <span class="se">\
</span><span class="se"></span>sway
</code></pre></div><p>Then, on my laptop, I can run this script:</p>
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh"><span class="cp">#!/bin/sh
</span><span class="cp"></span><span class="k">if</span> <span class="o">[</span> <span class="nv">$#</span> -eq <span class="m">0</span> <span class="o">]</span>
<span class="k">then</span>
	<span class="nb">export</span> <span class="nv">XDG_RUNTIME_DIR</span><span class="o">=</span>/tmp
	<span class="nb">exec</span> cage sway-remote launch
<span class="k">else</span>
	sleep <span class="m">3</span>
	<span class="nb">exec</span> xfreerdp <span class="se">\
</span><span class="se"></span>		-v homura <span class="se">\
</span><span class="se"></span>		--bpp <span class="m">32</span> <span class="se">\
</span><span class="se"></span>		--size 1280x800 <span class="se">\
</span><span class="se"></span>		--rfx
<span class="k">fi</span>
</code></pre></div><p>The first branch is taken on the first run, and it starts up cage and asks it to
run this script again. The second branch then starts up xfreerdp and connects to
my desktop (its hostname is <code>homura</code>). xfreerdp is then fullscreened and all of
my laptop&rsquo;s input events are directed to it. The result is an experience which
is basically identical to running sway directly on my laptop, except it&rsquo;s
actually running on my desktop and using the remote desktop protocol to send
everything back and forth.</p>
<p>This isn&rsquo;t especially practical, but it is a cool hack. It&rsquo;s definitely not
network transparency like some people want, but I wasn&rsquo;t aiming for that. It&rsquo;s
just a neat thing you can do now that we have an RDP backend for wlroots. And
congrats again to Jente - be sure to give Cage a look and see if you can think
of any other novel use-cases, too!</p>


guid: https://drewdevault.com/2019/04/29/Shut-up-and-get-back-to-work-style.html
title: The "shut up and get back to work" coding style guide
url: https://drewdevault.com/2019/04/29/Shut-up-and-get-back-to-work-style.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>So you&rsquo;re starting a new website, and you open the first CSS file. What style do
you use? Well, you hate indenting with spaces passionately. You know tabs are
right because they&rsquo;re literally made for this, and they&rsquo;re only one byte, and
these god damn spaces people with their bloody spacebars&hellip;</p>
<p>Shut up and use spaces. That&rsquo;s how CSS is written<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. And you, mister web
programmer, coming out of your shell and dipping your toes into the world of
Real Programming, writing your first Golang program: use tabs, jerk. There&rsquo;s
only one principle that matters in coding style: don&rsquo;t rock the boat. Just do
whatever the most common thing is in the language you&rsquo;re working in. Write your
commit messages the same way as everyone else, too. Then shut up and get back to
work. This hill isn&rsquo;t worth dying on.</p>
<p>If you&rsquo;re working on someone else&rsquo;s project, this goes double. Don&rsquo;t get snippy
about their coding style. Just follow their style guide, and if there isn&rsquo;t one,
just make your code look like the code around it. It&rsquo;s none of your goddamn
business how they choose to style their code.</p>
<p>Shut up and get back to work.</p>
<p>Ranting aside, seriously - which style guide you use doesn&rsquo;t matter nearly as
much as using one. Just pick the one which is most popular or which is already
in use by your peers and roll with it.</p>
<div style="margin-bottom: 5rem"></div>
<p>&hellip;though since I&rsquo;m talking about style anyway, take a look at this:</p>
<div class="highlight"><pre class="chroma"><code class="language-c" data-lang="c"><span class="k">struct</span> <span class="n">wlr_surface</span> <span class="o">*</span><span class="nf">wlr_surface_surface_at</span><span class="p">(</span><span class="k">struct</span> <span class="n">wlr_surface</span> <span class="o">*</span><span class="n">surface</span><span class="p">,</span>
                                           <span class="kt">double</span> <span class="n">sx</span><span class="p">,</span> <span class="kt">double</span> <span class="n">sy</span><span class="p">,</span>
                                           <span class="kt">double</span> <span class="o">*</span><span class="n">sub_x</span><span class="p">,</span> <span class="kt">double</span> <span class="o">*</span><span class="n">sub_y</span><span class="p">)</span> <span class="p">{</span>
    <span class="c1">// Do stuff
</span><span class="c1"></span><span class="p">}</span>
</code></pre></div><p>There&rsquo;s a lot of stupid crap which ends up in style guides, but this is by far
the worst. Look at all that wasted whitespace! There&rsquo;s no room to write your
parameters on the right, and you end up with 3 lines where you could have two.
And you have to mix spaces and tabs! God dammit! This is how you should do it:</p>
<div class="highlight"><pre class="chroma"><code class="language-c" data-lang="c"><span class="k">struct</span> <span class="n">wlr_surface</span> <span class="o">*</span><span class="nf">wlr_surface_surface_at</span><span class="p">(</span><span class="k">struct</span> <span class="n">wlr_surface</span> <span class="o">*</span><span class="n">surface</span><span class="p">,</span>
        <span class="kt">double</span> <span class="n">sx</span><span class="p">,</span> <span class="kt">double</span> <span class="n">sy</span><span class="p">,</span> <span class="kt">double</span> <span class="o">*</span><span class="n">sub_x</span><span class="p">,</span> <span class="kt">double</span> <span class="o">*</span><span class="n">sub_y</span><span class="p">)</span> <span class="p">{</span>
    <span class="c1">// Do stuff
</span><span class="c1"></span><span class="p">}</span>
</code></pre></div><p>Note the extra indent to distinguish the parameters from the body and the
missing garish hellscape of whitespace. If you do this in your codebase, I&rsquo;m not
going to argue with you about it, but I am going to have to talk to my therapist
about it.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>For the record, tabs are objectively better. Does that mean I&rsquo;m going to write my JavaScript with tabs? Hell no! <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/05/01/Announcing-wio.html
title: Announcing Wio: A clone of Plan 9's Rio for Wayland
url: https://drewdevault.com/2019/05/01/Announcing-wio.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>For a few hours here and there over the past few months, I&rsquo;ve been working on a
side project: <a href="https://wio-project.org">Wio</a>. I&rsquo;ll just let the (3 minute)
screencast do the talking first:</p>
<p><video src="https://yukari.sr.ht/wio.webm" controls></video></p>
<p><strong>Note</strong>: this video begins with several seconds of grey video. This is normal.</p>
<p>In short, Wio is a Wayland compositor based on wlroots which has a similar look
and feel to Plan 9&rsquo;s Rio desktop. It works by running each application in its
own nested Wayland compositor, based on <a href="https://www.hjdskes.nl/projects/cage/">Cage</a> - yet another wlroots-based
Wayland compositor. I used Cage in <a href="https://drewdevault.com/2019/04/23/Using-cage-for-a-seamless-RDP-Wayland-desktop.html">last week&rsquo;s RDP article</a>, but
here&rsquo;s another cool use-case for it.</p>
<p>The behavior this allows for (each window taking over its parent&rsquo;s window,
rather than spawning a new window) has been something I wanted to demonstrate on
Wayland for a very long time. This is a good demonstration of how Wayland&rsquo;s
fundamentally different and conservative design allows for some interesting
use-cases which aren&rsquo;t possible at all on X11.</p>
<p>I&rsquo;ve also given Wio some nice features which are easy thanks to wlroots, but
difficult on Plan 9 without kernel hacking. Namely, these are multihead support,
HiDPI support, and support for the wlroots layer shell protocol. Several other
wlroots protocols were invited to the party, useful for taking screenshots,
redshift, and so on. Layer shell support is particularly cool, since programs
like swaybg and waybar work on Wio.</p>
<p>In terms of Rio compatability, Wio has a ways to go. I would seriously
appreciate help from users who are interested in improving Wio. Some notably
missing features include:</p>
<ul>
<li>Any kind of filesystem resembling Rio&rsquo;s window management filesystem. In
theory this ought to be do-able with FUSE, at least in part (/dev/text might
be tough).</li>
<li>Running every application in its own namespace, for double the Plan 9</li>
<li>Hiding/showing windows (that menu entry is dead)</li>
<li>Joint improvements with Cage to bring greater support for Wayland features,
like client-side window resize/move, fullscreen windows, etc</li>
<li>Damage tracking to avoid re-rendering everything on every frame, saving
battery life and GPU time</li>
</ul>
<p>If you&rsquo;re interested in helping, please join the IRC channel and say hello:
<a href="http://webchat.freenode.net/?channels=%23wio&amp;uio=MTA9dHJ1ZSYxMT0xNzQmMTM9ZmFsc2U4c">#wio on irc.freenode.net</a>. For Wio&rsquo;s source code and other
information, visit the website at <a href="https://wio-project.org">wio-project.org</a>.</p>


guid: https://drewdevault.com/2019/05/06/Calculate-your-doation-fees-for-Patreon.html
title: Calculating your donation's value following Patreon's fee changes
url: https://drewdevault.com/2019/05/06/Calculate-your-doation-fees-for-Patreon.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>In January 2018, I wrote a blog post which included a <a href="https://drewdevault.com/2018/01/16/Fees-on-donation-platforms.html">fee
calculator</a>.
Patreon <a href="https://www.patreon.com/new-creator-plans">changes their fee model
tomorrow</a>, and it&rsquo;s time for an
updated calculator. I&rsquo;m grandfathered into the old fees, so not much has changed
for me, but I want to equip Patreon users - creators and supporters - with more
knowledge of how their money is moving through the platform.</p>
<p>Patreon makes money by siphoning some money off the top of a donation flow
between supporters and creators. Because of the nature of its business (a
private, VC-backed corporation), the siphon&rsquo;s size and semantics are prone to
change in undesirable ways, since VC&rsquo;s expect infinite growth and a private
business generally puts profit first. For this reason, I diversify my income, so
that when Patreon makes these changes it limits their impact on my financial
well-being.  Even so, Patreon is the biggest of my donation platforms,
representing over $500/month at the time of writing (<a href="https://drewdevault.com/donate/">full breakdown
here</a>)<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>.</p>
<p>So, for any patrons who are curious about where their money goes, here&rsquo;s a handy
calculator to help you navigate the complex fees. Enjoy!</p>
<p><strong>Note</strong>: I don&rsquo;t normally ask you to share my posts, but the Patreon community
is too distributed for me to effectively reach them alone. Please share this
with your Patreon creators and communities!</p>
<p><noscript>Sorry, the calculator requires JavaScript.</noscript></p>
<div id="react-root"></div>
<script src="https://drewdevault.com/js/donation-calc.js"></script>
<p><strong>Note</strong>: this calculator does not include the withdrawal fee. When the creator
withdraws their funds from the platform, an additional fee is charged, but the
nature of that fee changes depending on the frequency with which they make
withdrawals and the total amount of money they make from all patrons - which is
information that&rsquo;s not easily available to the average patron for using with
this calculator. For details on the withdrawal fees, see <a href="https://support.patreon.com/hc/en-us/articles/203913489-What-are-my-options-to-receive-payout-">Patreon&rsquo;s support
article on the
subject</a>.</p>
<p>One question that&rsquo;s been left unanswered is how many times Patreon is going to
charge patrons for each creator they support. Previously, they batched payments
and only accordingly charged the payment processing fees once. However, along
with these changes, they&rsquo;re going to charge payment processing fees for each
creator, but they haven&rsquo;t lowered the payment processing fees. When we take a
look at our bank returns in the coming months, if Patreon is still batching
payments internally&hellip; hmm, where is the extra money going? We&rsquo;ll have to wait
and see.</p>
<h2 id="founding-creators">What are founding creators?</h2>
<p>Creators who used the Patreon platform prior to 2019-05-07 are &ldquo;founding
creators&rdquo;, and have different rates. They have different rates for each plan,
and lower payment processing fees. Founding creators are also not usually lite
creators, but were grandfathered into the pro plan.</p>
<h2 id="charge-up-front">What does charge up front mean?</h2>
<p>Some creators have the option to charge you as soon as you join the platform,
rather than once monthly or per-creation. This results in higher payment
processing fees for founding creators, as Patreon cannot batch the charge
alongside with your other creators.</p>
<h2 id="which-plan">How do I know what plan my creator uses?</h2>
<p>We can guess which plan our creator uses by looking at the features they use on
Patreon. Here are some giveaways:</p>
<ul>
<li>If they have different membership tiers, they use the Pro plan or better.</li>
<li>If they offer merch through Patreon, they use the Premium plan.</li>
</ul>
<p>You can also just reach out to your creator and ask!</p>
<!-- Hack to get footnotes from javascript to work -->
<p><span style="display: none"><sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup></span></p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>This is supplemented with my Sourcehut income as well, which is covered in the recent <a href="https://lists.sr.ht/~sircmpwn/sr.ht-discuss/%3C20190426160729.GC1351@homura.localdomain%3E">Q1 financial report</a>, as well as some <a href="https://drewdevault.com/consulting">consulting work</a>, which I don&rsquo;t publish numbers for. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>This is an assumption based on public PayPal and Stripe payment processing rates. In practice, it&rsquo;s likely that Patreon has a volume discount with their payment processors. Patreon does not publish these rates. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/05/13/Git-email-webcast.html
title: Webcast: Reviewing git & mercurial patches with email
url: https://drewdevault.com/2019/05/13/Git-email-webcast.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>With the availability of new resources like
<a href="https://git-send-email.io">git-send-email.io</a>, I&rsquo;ve been working on making the
email-based workflow more understandable and accessible to the world. One thing
that&rsquo;s notably missing from this tutorial, however, is the maintainer side of
the work. I intend to do a full write-up in the future, but for now I thought
it&rsquo;d be helpful to clarify my workflow a bit with a short webcast. In this
video, I narrate my workflow as I review a few
<a href="https://sourcehut.org">sourcehut</a> patches and participate in some dicsussions.</p>
<video src="https://yukari.sr.ht/git-screencast.webm" controls>
  Your browser does not support HTML5 video, or webm video.
</video>
<p>Links:</p>
<ul>
<li><a href="http://www.mutt.org/">mutt</a>: my email client</li>
<li><a href="https://git.sr.ht/~sircmpwn/dotfiles/tree/master/.config/mutt/muttrc">my personal mutt config</a></li>
<li><a href="https://git.sr.ht/~sircmpwn/dotfiles/tree/master/bin/semver">my &ldquo;semver&rdquo; script</a></li>
</ul>
<p>Also check out <a href="https://git.sr.ht/~sircmpwn/aerc2">aerc</a>, a replacement for mutt
that I&rsquo;ve been working on over the past year or two. I will be writing more
about that project soon.</p>


guid: https://drewdevault.com/2019/05/15/Status-update-May-2019.html
title: Status update, May 2019
url: https://drewdevault.com/2019/05/15/Status-update-May-2019.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>This month, it seems the most exciting developments again come from the realm of
email. I&rsquo;ve got cool email-related news to share for aerc, lists.sr.ht, and
todo.sr.ht, and many cool developments in my other projects to share.</p>
<p>Let&rsquo;s start with lists.sr.ht: I have broken ground on the web-based patch review
tools! I promised these features when I started working on sourcehut, to make
the email-based workflow more enticing to those who would rather work on the
web. Basically, this gives us a Github or Gerrit-esque review UI for patches
which arrive on the mailing list. Thanks to <a href="https://git.sr.ht/~emersion/python-emailthreads">a cool
library</a> Simon Ser wrote for
me&hellip; almost a year ago&hellip; I&rsquo;m able to take a thread of emails discussing a
patch and organically convert them into inline feedback on the web.</p>
<p><a href="https://lists.sr.ht/~philmd/qemu/patches/5556"><img src="https://sr.ht/sjtE.png" alt=""></a></p>
<small style="display: block; text-align: center;">
  Click the screenshot to visit this page on the web
</small>
<p>This is generated from organic discussions where the participants don&rsquo;t have to
do anything special to participate - in the discussion this screenshot is
generated from, the participants aren&rsquo;t even aware that this process is taking
place. This approach allows users who prefer a web-based workflow to interact
with traditional email-based patch review seamlessly. Future improvements will
include detecting new revisions of a patch, side-by-side diff and diffs between
different versions of a patch, and using the web interface to review a patch -
which will generate an email on the list. I&rsquo;d also like to extend git.sr.ht with
web support for git send-email, allowing you to push to your git repo and send a
patch off to the mailing list from the web. It should also be possible to
combine this with dispatch.sr.ht to have bidirectional code reviews between
mailing lists and Github, Gitlab, etc - with no one on either side being any the
wiser to the preferred workflow of the other.</p>
<p>In other exciting email-related news, aerc2 now supports composing emails -
a feature which has been a long time coming, and was not even present in aerc1!
Check it out:</p>
<script
  id="asciicast-CqTukJZoTq7ZgPmsjhIbQyUjb"
  src="https://asciinema.org/a/pafXXANiWHY9MOH2yXdVHHJRd.js" async
></script>
<p>Outgoing email configuration supports SMTP, STARTTLS, and SMTPS, with sendmail
support planned. Outgoing emails are edited with our embedded terminal emulator
using vim, or your favorite <code>$EDITOR</code>. Still to come: replying to emails &amp; PGP
support. I could use your help here! If you want a chance to write some cool Go
code, stop by the IRC channel and say hello: <a href="http://webchat.freenode.net/?channels=aerc&amp;uio=d4">#aerc on
irc.freenode.net</a>. Once aerc
matures a little bit, I also want to start working on a git integration which
will continue making email an even more compelling platform for software
development.</p>
<p>Let&rsquo;s talk about Wayland next. I&rsquo;ve been shipping release candidates for sway
1.1 - <a href="https://github.com/swaywm/sway/issues/3861#issuecomment-487073065">check out the provisional changelog
here</a>. The
highlights are probably the ability to inhibit idle with arbitrary criteria, and
touch support for swaybar. The release candidates have been pretty quiet - we
might end up shipping this as early as rc4. wlroots 0.6.0 was also released,
though for end-users it doesn&rsquo;t include much. We&rsquo;ve removed the long-deprecated
wl_shell, and have made plans to start removing other deprecated protocols. I&rsquo;ve
also been working with the broader Wayland community on establishing a
governance model for protocol standardization - <a href="https://lists.freedesktop.org/archives/wayland-devel/2019-May/040532.html">read the latest draft
here</a>.</p>
<p>I&rsquo;ve also started working on a Wayland book! It&rsquo;s intended as a comprehensive
reference on the Wayland protocol, useful for authors hoping to write both
Wayland compositors and Wayland clients. It does not go into all of the
nitty-gritty details necessary for writing a Wayland compositor for Linux (that
is, the sort of knowledge necessary for using wlroots, or even making wlroots
itself), but that&rsquo;ll be a task for another time. Instead, I focus on the Wayland
protocol itself, explaining how the wire protocol works and the purpose and
usage of each interface in <code>wayland.xml</code>, as well as <code>libwayland</code>. I intend to
sell this book, but when you buy it you&rsquo;ll receive a DRM-free CC-NC-ND copy that
you can share freely with your friends.</p>
<p>Before I move on from Wayland news, also check out
<a href="https://wio-project.org/">Wio</a> if you haven&rsquo;t yet - I wrote a blog post about
it <a href="https://drewdevault.com/2019/05/01/Announcing-wio.html">here</a>. In short: I
made a novel new Wayland compositor in my spare time which behaves like plan 9&rsquo;s
Rio.  See the blog post for more details!</p>
<p>Following the success of <a href="https://git-send-email.io">git-send-email.io</a>, I
published a similar website last week: <a href="https://git-rebase.io">git-rebase.io</a>.
The purpose of this website is to teach readers how to use git rebase,
explaining how to use its primitives to accomplish common high-level tasks in a
way that leaves the reader equipped to apply those primitives to novel
high-level tasks in the course of their work. I hope you find it helpful! I&rsquo;ve
also secured git-filter-branch.io and git-bisect.io to explain additional
useful, but confusing git commands in the future.</p>
<p>Brief updates for other projects: I&rsquo;ve been ramping up RISC-V work again,
helping Golang test their port, testing out u-Boot, and working on the Alpine
port some more. cozy has seen only a little progress, but the parser is
improving and it&rsquo;s now emitting a (very incomplete) AST for source files you
feed to it. Godot is on hold pending additional upstream bandwidth for code
review.</p>
<p>That&rsquo;s all for today! Thank you so much for your support. It&rsquo;s pretty clear by
now that my productivity is way higher now that I&rsquo;m able to work full-time on
open source, thanks to your support. I&rsquo;ll see you for next month&rsquo;s update!</p>
<small class="text-muted">
This work was possible thanks to users who support me financially. Please
consider <a href="https://drewdevault.com/donate">donating to my work</a> or <a
href="https://sourcehut.org">buying a sourcehut.org subscription</a>. Thank you!
</small>


guid: https://drewdevault.com/2019/05/24/What-is-a-fork.html
title: What is a fork, really, and how GitHub changed its meaning
url: https://drewdevault.com/2019/05/24/What-is-a-fork.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>The fork button on GitHub - with the little number next to it for depositing
dopamine into your brain - is a bit misleading. GitHub co-opted the meaning of
&ldquo;fork&rdquo; to trick you into participating in their platform more. They did this in
a well-intentioned way, for the sake of their pull requests feature, but
ultimately this design is self-serving and causes some friction when
contributors venture out of their GitHub sandbox and into the rest of the
software development ecosystem. Let&rsquo;s clarify what &ldquo;fork&rdquo; really means, and what
we do without GitHub&rsquo;s concept of one - for it is in this difference that we
truly discover how git is a <em>distributed</em> version control system.</p>
<p><strong>Disclaimer</strong>: I am the founder of <a href="https://sourcehut.org">SourceHut</a>, a
product which competes with GitHub and embraces the &ldquo;bazaar<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>&rdquo; model described
in this article.</p>
<p>On GitHub, a fork refers to a copy of a repository used by a contributor<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> to
stage changes they&rsquo;d like to propose upstream. Prior to GitHub (and in many
places still today), we&rsquo;d call such a repository a &ldquo;personal branch&rdquo;. A personal
branch doesn&rsquo;t need to be published to be useful - you can just <code>git clone</code> it
locally and make your changes there without pushing them to a public, hosted
repository. Using <a href="https://git-send-email.io">email</a>, you can send changes from
your local, unpublished repository for consideration upstream. Outside of
GitHub and its imitators, most contributors to a project don&rsquo;t have a published
version of their repository online at all, skipping that step and saving some
time.</p>
<p>In some cases, however, it&rsquo;s useful to publish your personal branch online. This
is often done when a team of people is working on a long-lived branch to later
propose upstream - for example, I&rsquo;ve been doing this while working on the RISC-V
port of musl libc. It gives us a space to collaborate and work while preparing
changes which will eventually be proposed upstream, as well as a place for
interested testers to obtain our experimental work to try themselves. This is
also done by individuals, such as Greg Kroah-Hartman&rsquo;s Linux branches, which are
useful for testing upcoming changes to the Linux kernel.</p>
<p>Greg is not alone in publishing a repo like this. In fact, there are <a href="https://git.kernel.org/pub/scm/linux/kernel/git/">hundreds of
kernel trees like this</a>. These act as staging areas for long-term
workstreams, or for the maintainers of many subsystems of the kernel.  Changes
in these repositories gradually flow upwards towards the &ldquo;main&rdquo; tree,
<a href="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/">torvalds/linux</a>. The precise meaning of &ldquo;linux&rdquo; is rather loose
in this context. An argument could be made that torvalds/linux is Linux, but
that definition wouldn&rsquo;t capture the LTS branches. Many distros also apply their
own patches on top of Torvalds, perhaps sourcing them from the maintainers of
drivers they need a bugfix for, or they maintain their own independent trees
which periodically pull in lump sums of changes from other trees - meaning that
the simple definition might not include the version of Linux which is installed
on your computer, either. This ambiguity is a feature - each of these trees is a
valid definition of Linux in its own right.</p>
<p>This is the sense in which git is &ldquo;distributed&rdquo;. The idea of a canonical
upstream is not written in stone in the way that GitHub suggests it might be.
After all, open-source software is a collaborative endeavour. What makes Jim&rsquo;s
branch more important that John&rsquo;s branch? John&rsquo;s branch is definitely more
important if it has the bugfixes you need. In fact, your branch, based on Jim&rsquo;s,
with some patches cherry-picked from John, and a couple of fixes of your own
mixed in, may in fact be the best version of the software for you.</p>
<p>This is how the git community gets along without the GitHub model of &ldquo;forks&rdquo;.
This design has allowed the largest and most important projects in the world to
flourish, and git was explicitly designed around this model. We refer to this as
the &ldquo;bazaar&rdquo; model, the metaphor hopefully being fairly obvious at this point.
There is another model, which GitHub embodies instead: the cathedral. In this
model, the project has a central home and centralized governance, run by a small
number of people. The cathedral doesn&rsquo;t necessarily depend on the GitHub idea of
&ldquo;forks&rdquo; and pull requests - that is, you can construct a cathedral with
email-driven development or some other model - but on GitHub the bazaar option
is basically absent.</p>
<p>In the introduction I said that GitHub attempts to replace an existing meaning
for &ldquo;fork&rdquo;. So what does forking actually mean, then? Consider a project with
the cathedral model. What happens when there&rsquo;s a schism in the church? The
answer is that some of the contributors can take the code, put up a new branch
somewhere, and stake a flag in the ground. They rename it and commit to
maintaining it entirely independently of the original project, and encourage
contributors, new and old alike, to abandon the old dogma in favor of theirs.
At this point, the history<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup> begins to diverge. The new contingent pulls in
all of the patches that were denied upstream and start that big refactoring to
mold it in their vision. The project has been <strong>forked</strong>. A well known example
is when ffmpeg was forked to create libav.</p>
<p>This is usually a traumatic event for the project, and can have repercussions
that last for years. The precise considerations that should go into forking a
project, these repercussions and how to address them, and other musings are
better suited for a separate article. But this is what &ldquo;fork&rdquo; meant before
GitHub, and this meaning is still used today - albeit more ambiguously.</p>
<p>If &ldquo;fork&rdquo; already had this meaning, why did GitHub adopt their model? The
answer, as it often will be, is centralization of power. GitHub is a
proprietary, commercial service, and their ultimate goal is to turn a profit.
The design of GitHub&rsquo;s fork and pull request model creates a cathedral that
keeps people on their platform in a way that a bazaar would not. A distributed
version control system like git, built on a distributed communications protocol
like email, is hard to disrupt with a centralized service. So GitHub designed
their own model.</p>
<p>As a parting note, I would like to clarify that this isn&rsquo;t a condemnation of
GitHub. I still use their service for a few projects, and appreciate the
important role GitHub has played in the popularization of open source. However,
I think it&rsquo;s important to examine the services we depend on, to strive to
understand their motivations and design. I also hope the reader will view the
software ecosystem through a more interesting lens for having read this article.
Thank you for reading!</p>
<hr>
<p><strong>P.S.</strong> Did you know that GitHub also captured the meaning of &ldquo;pull request&rdquo;
from git&rsquo;s own <a href="https://www.git-scm.com/docs/git-request-pull">request-pull</a>
tool? git request-pull prepares an email which will ask the recipient to fetch
changes from a public repository and integrate them into their own branch. This
is used when a patch is insufficient - for example, when Linux subsystem
maintainers want to ship a large group of changes to Torvalds for the next
kernel release. Again, the original version is distributed and bazaar-like,
whereas GitHub&rsquo;s is centralized and makes you stay on their platform.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Not the bazaar version control system, but bazaar the concept. This is explained later in the article. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>And by bots to increase their reputation, and by confused users who don&rsquo;t know what the button means. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>Git history in particular, but also the other kind. <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


guid: https://drewdevault.com/2019/06/03/Announcing-aerc-0.1.0.html
title: Initial pre-release of aerc: an email client for your terminal
url: https://drewdevault.com/2019/06/03/Announcing-aerc-0.1.0.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>After years of painfully slow development, the <a href="https://aerc-mail.org">aerc email
client</a> has seen a huge boost in its pace of development
recently. This leads to today&rsquo;s announcement: <a href="https://git.sr.ht/~sircmpwn/aerc/refs/0.1.0">aerc 0.1.0 is now
available</a>! After my transition to <a href="https://drewdevault.com/2019/01/15/Im-doing-FOSS-full-time.html">working on free software full
time</a> allowed me to spend more time on more projects, I was able
to invest considerably more time into aerc. Your support led us here: thank you
to all of the people who <a href="https://drewdevault.com/donate">donate to my work</a>!</p>
<p>I&rsquo;ve prepared a short webcast demonstrating aerc&rsquo;s basic features - give it a
watch if you&rsquo;re curious about what aerc looks like &amp; what makes it interesting.</p>
<video controls>
  <source src="https://yukari.sr.ht/aerc-intro.webm"></source>
  <source src="https://yukari.sr.ht/aerc-intro.mp4"></source>
  A video would be shown here, but your web browser does not support it.
</video>
<p>In summary, aerc is an email client which runs in your terminal emulator. If
you&rsquo;re coming from mutt, you&rsquo;ll appreciate its more efficient &amp; reliable
networking, a keybinding system closer to vims, and embedded terminal emulator
allowing you to compose emails and read new ones at the same time. It builds on
this foundation with a lot of new and exciting features. For example, its
&ldquo;filter&rdquo; feature allows us to review patches with syntax highlighting:</p>
<p><img src="https://sr.ht/JoqH.png" alt="Screenshot of aerc displaying a patch"></p>
<p>The embedded terminal emulator also allows us convenient access to nearby git
repositories for running tests against incoming patches, pushing the changes
once accepted, or anything else you might want to do. Want to run
<a href="https://weechat.org/">Weechat</a> in an aerc tab? Just like that, aerc has a chat
client! Writing emails in vim, manipulating git &amp; hg repositories, playing
nethack to kill some time&hellip; all stuff you never realized your email client was
missing.</p>
<p>I plan on extending aerc in the future with more integrations with version
control systems, calendar &amp; contacts support, and more email configurations like
notmuch and JMAP. Please consider
<a href="https://git.sr.ht/~sircmpwn/aerc">contributing</a> if you&rsquo;re interested in writing
a little Go, or <a href="https://drewdevault.com/donate">donating monthly</a> to ensure I
always have time to work on this and other free software projects. Give aerc a
try and let me know what you think!</p>


guid: https://drewdevault.com/2019/06/13/My-journey-from-MIT-to-GPL.html
title: My personal journey from MIT to GPL
url: https://drewdevault.com/2019/06/13/My-journey-from-MIT-to-GPL.html
feedurl: https://drewdevault.com/blog/index.xml
content:
<p>As I got started writing open source software, I generally preferred the MIT
license. I actually made fun of the &ldquo;copyleft&rdquo; GPL licenses, on the grounds that
they are less free. I still hold this opinion today: the GPL license is less
free than the MIT license - but today, I believe this in a good way.</p>
<p>If you haven&rsquo;t yet, I suggest reading the <a href="https://opensource.org/licenses/MIT">MIT
license</a> - it&rsquo;s very short. It satisfies
the four essential freedoms guaranteed of <a href="https://www.gnu.org/philosophy/free-sw.html">free
software</a>:</p>
<ol>
<li>The right to use the software for any purpose.</li>
<li>The right to study the source code and change it as you please.</li>
<li>The right to redistribute the software to others.</li>
<li>The right to distribute your modifications to the software.</li>
</ol>
<p>The MIT license basically allows you to do whatever you want with the software.
It&rsquo;s one of the most hands-off options: &ldquo;here&rsquo;s some code, you can do anything
you want with it.&rdquo; I favored this because I wanted to give users as much freedom
to use my software as possible. The GPL, in addition to being a <a href="https://www.gnu.org/licenses/gpl-3.0.html">much more
complex tome to understand</a>, is more
restrictive. The GPL forces you to use the GPL for derivative works as well.
Clearly this affords you less freedom to use the software. Obligations are the
opposite of freedoms.</p>
<p>When I first got into open source, I was still a Windows user. As I gradually
waded deeper and deeper into the free software pond, I began to use Linux more
often<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. Even once I started using Linux as my daily driver, however, it took
a while still for the importance of free software to set in. But this
realization is inevitable, for a programmer immersed in Linux. It radically
changes your perspective when all of the software you use guarantees these four
freedoms. If I&rsquo;m curious about how something works, I can usually be reading the
code within a few seconds. I can find the author&rsquo;s name and email in the git
blame and shoot them some questions. And when I find a bug, I can fix it and
send them a patch.</p>
<p>The weight of these possibilities did not occur to me immediately, instead
slowly becoming evident over time. Today, this cycle is almost muscle memory.
Pulling down source, grepping for files related to an itch I need to scratch,
compiling and installing the modified version, and sending my work upstream -
it&rsquo;s become second nature to me. These days, on the rare occasion that I run
into some proprietary software, this all grinds to a halt. It&rsquo;s like miscounting
the number of steps on your staircase in the dark. These moments drive the truth
home: Free software is good. It&rsquo;s starkly better than the alternative. And
copyleft defends it. Now that I&rsquo;ve had a taste, you bet your ass I&rsquo;m not going
to give it up.</p>
<p>As the number of hours I&rsquo;ve spent on FOSS projects grew from tens of hours, to
hundreds, to thousands and tens of thousands, I&rsquo;ve learned that the effort I
sink into my work far outstrips the effort required to reuse my work. The
collective effort of the free software community amounts to tens of millions of
hours of work, which you can download at touch of a button, for free. If the
people with their fingers on that button held these same ideals, we wouldn&rsquo;t
need the GPL. The reality, however, is that we live in a capitalist world. Our
socialist free software utopia is ripe for exploitation by capitalists, and
they&rsquo;ll be rewarded for doing so. Capitalism is about enriching yourself - not
enriching your users and certainly not enriching society.</p>
<p>Your parents probably taught you about the Golden Rule when you were young: do
unto others as you would have them do unto you. The GPL is the legal embodiment
of this Golden Rule: in exchange for benefiting from my hard work, you just have
to extend me the same courtesy. Its the unfortunate acknowledgement that we&rsquo;ve
created a society that incentivises people to forget the Golden Rule. I give
people free software because I want them to reciprocate with the same. That&rsquo;s
really all the GPL does. Its restrictions just protect the four freedoms in
derivative works. Anyone who can&rsquo;t agree to this is looking to exploit your work
for their gain - and definitely not yours.</p>
<p>I don&rsquo;t plan on relicensing my historical projects, but my new projects have
used the GPL family of licenses for a while now. I think you should seriously
consider it as well.</p>
<section class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1" role="doc-endnote">
<p>Fun fact: the first time I used Linux was as a teenager, in order to get around the internet filtering software my parents had installed on our Windows PC at home. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p>
</li>
</ol>
</section>


