106 lines
4.4 KiB
HTML
106 lines
4.4 KiB
HTML
<!doctype html>
|
|
<html>
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<meta http-equiv="x-ua-compatible" content="ie=edge">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
|
|
|
<meta name="theme-color" content="#00cc00">
|
|
<link rel="icon" type="image/png" sizes="192x192" href="https://tilde.team/apple-touch-icon-precomposed.png">
|
|
<link rel="icon" type="image/png" sizes="96x96" href="https://tilde.team/favicon-96x96.png">
|
|
|
|
<link rel="stylesheet" href="https://tilde.team/~ben/gruvbox/gruvbox.css">
|
|
<link rel="stylesheet" href="extra.css">
|
|
|
|
<link rel="alternate" type="application/rss+xml" title="subscribe to this page..." href="feed.rss" />
|
|
<title>networking nonsense</title>
|
|
</head><body>
|
|
<main>
|
|
|
|
<div id="divbodyholder">
|
|
<div class="headerholder"><div class="header">
|
|
<div id="title">
|
|
<h1 class="nomargin"><a class="ablack" href="https://tilde.team/~ben/blog/index.html">blog // ~ben</a></h1>
|
|
<div id="description">a blog about tildes and other things</div>
|
|
</div></div></div>
|
|
<div id="divbody"><div class="content">
|
|
<!-- entry begin -->
|
|
<h3><a class="ablack" href="networking-nonsense.html">
|
|
networking nonsense
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201902110240.20# -->
|
|
<div class="subtitle">February 11, 2019 —
|
|
~ben
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<p>i've recently been working on setting up <a href="https://drone.tildegit.org">drone ci</a>
|
|
on the tilde.team machine. however, there's been something strange going on
|
|
with the networking on there.</p>
|
|
|
|
<hr />
|
|
|
|
<p>starting up drone with <a href="https://tildegit.org/tildeverse/drone/src/branch/master/docker-compose.yml">docker-compose</a>
|
|
didn't seem to be working: <code>netstat -tulpn</code> showed the port binding properly
|
|
to 127.0.0.1:8888 but i was completely unable to get anything from it (using
|
|
curl the nginx proxy that was to come).</p>
|
|
|
|
<p>i ended up scrapping docker on the ~team box itself and moving it into a lxd
|
|
container (pronounced "lex-dee") with nesting enabled.</p>
|
|
|
|
<p>this got us in to another problem that had been seen before when using nginx
|
|
to proxy to apps running in other containers. requests were dropped
|
|
intermittently, sometimes hanging for upwards of 30 seconds.</p>
|
|
|
|
<p>getting frustrated with this error, i tried to reproduce it on another host.
|
|
both the docker-proxy and nginx->lxd proxies work on the first try yielded no
|
|
clues as to where things were going wrong.</p>
|
|
|
|
<p>in a half-awake stupor last saturday evening, i decided to try rule out ipv6
|
|
by disabling it system-wide. as is expected for sleepy work, it didn't fix the
|
|
problem and created more in the process. </p>
|
|
|
|
<p>feeling satisfied that the problem didn't lie with ipv6, i re-enabled it, only
|
|
to find that i was unable to bind nginx to my allocated /64. i may or may not
|
|
have ranted a bit about this on irc but i was able to get it back up and
|
|
running by restarting systemd-networkd.</p>
|
|
|
|
<p>one step forwards broke something and now we're back to where we started with
|
|
the original problem of the intermittent hangups to the lxd container.</p>
|
|
|
|
<p>seeing my troubles on irc, <a href="https://tilde.team/~jchelpau/">jchelpau</a> offered
|
|
to help dig in to the problem with a a fresh set of eyes. he noted right away
|
|
that pings over ipv6 to the containers worked fine, but ipv4 did not.</p>
|
|
|
|
<p>we ended up looking at the firewall configurations, only to find that one of
|
|
the subnets i blocked after november's <a href="november-13-post-mortem.html">nmap incident</a>
|
|
included lxdbr0's subnet (the bridge device used by lxd).</p>
|
|
|
|
<p>now that i made the exeption for lxdbr0, everything is working as expected!</p>
|
|
|
|
<p>thanks to <a href="https://tilde.team/~fosslinux/">fosslinux</a> and <a href="https://tilde.team/~jchelpau/">jchelpau</a>
|
|
for their debugging help!</p>
|
|
|
|
<p>tags: <a href='tag_tilde.html'>tilde</a>, <a href='tag_linux.html'>linux</a>, <a href='tag_sysadmin.html'>sysadmin</a></p>
|
|
|
|
|
|
|
|
|
|
<!-- text end -->
|
|
<!-- entry end -->
|
|
</div>
|
|
<div id="footer">CC by-nc-nd <a href="https://tilde.team/~ben/">~ben</a> — <a href="mailto:ben@tilde.team">ben@tilde.team</a><br>
|
|
generated with <a href="https://tildegit.org/team/bashblog">bashblog</a>, a single bash script to easily create blogs like this one</div>
|
|
</div></div>
|
|
<script src="https://utteranc.es/client.js"
|
|
repo="benharri/tilde"
|
|
issue-term="title"
|
|
crossorigin="anonymous"
|
|
theme="github-dark"
|
|
async>
|
|
</script>
|
|
|
|
</main>
|
|
<br>
|
|
</body></html>
|