295 lines
11 KiB
HTML
295 lines
11 KiB
HTML
<!doctype html>
|
|
<html>
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<meta http-equiv="x-ua-compatible" content="ie=edge">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
|
|
|
<meta name="theme-color" content="#00cc00">
|
|
<link rel="icon" type="image/png" sizes="192x192" href="https://tilde.team/apple-touch-icon-precomposed.png">
|
|
<link rel="icon" type="image/png" sizes="96x96" href="https://tilde.team/favicon-96x96.png">
|
|
|
|
<link rel="stylesheet" href="https://tilde.team/css/hacker.css">
|
|
<link rel="stylesheet" href="extra.css">
|
|
|
|
<link rel="alternate" type="application/rss+xml" title="subscribe to this page..." href="feed.rss" />
|
|
<title>blog // ~ben — posts tagged "linux"</title>
|
|
</head><body>
|
|
<div class="container">
|
|
|
|
<div id="divbodyholder">
|
|
<div class="headerholder"><div class="header">
|
|
<div id="title">
|
|
<h1 class="nomargin"><a class="ablack" href="https://tilde.team/~ben/blog/index.html">blog // ~ben</a></h1>
|
|
<div id="description">a blog about tildes and other things</div>
|
|
</div></div></div>
|
|
<div id="divbody"><div class="content">
|
|
<h3><a class="ablack" href="november-13-post-mortem.html">
|
|
november 13 post mortem
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201811132020.33# -->
|
|
<div class="subtitle">November 13, 2018 —
|
|
~ben
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<p>we had something of an outage on november 13, 2018 on tilde.team. </p>
|
|
|
|
<p>i awoke, not suspecting anything to be amiss. as soon as i logged in to check my email and irc mentions, it became clear.</p>
|
|
|
|
<p>tilde.team was at the least inaccessible, and at the worst, down completely. according the message in my inbox, there hade been an attempted "attack" from my IP.</p>
|
|
|
|
<blockquote>
|
|
<p>We have indications that there was an attack from your server.
|
|
Please take all necessary measures to avoid this in the future and to solve the issue.</p>
|
|
</blockquote>
|
|
|
|
<p>at this point, i have no idea what could have happened over night while i'm sleeping. the timestamp shows that it arrive only 30 minutes after i'd turned in for the night.</p>
|
|
|
|
<p>when i finally log on in the morning to check mails and irc mentions, i find that i'm unable to connect to tilde.team... strange, but ok; time to troubleshoot. i refresh the <a href="https://mail.tilde.team">webmail</a> to see what i'm missing. it ends up failing to find the server. even stranger! i'd better get the mails off my phone if they're on my @tilde.team mail!</p>
|
|
|
|
<p>here, i launch in to full debugging mode: what command was it? who ran it? </p>
|
|
|
|
<p>search <code>~/.bash_history</code> per user was not very successful. nothing i could find was related to net or map. i had checked <code>sudo grep nmap /home/*/.bash_history</code> and many other commands. </p>
|
|
|
|
<p>at this point, i had connected with other ~teammates across other irc nets (<a href="https://hashbang.sh/">#!</a>, <a href="https://tilde.town">~town</a>, etc). among suggestions to check <code>/var/log/syslog</code>, <code>/var/log/kern.log</code>, and <code>dmesg</code>, i finally decided to check <code>ps</code>. <code>ps -ef | grep nmap</code> yielded nmap on an obscured uid and gid, which is shortly established to belong to a container i had provisioned for <a href="/~fosslinux/">~fosslinux</a>.</p>
|
|
|
|
<p>i'm not considering methods of policing access to any site over port 80 and port 443. this is crazy. how do you police <code>nmap</code> when it isn't scanning on every port?</p>
|
|
|
|
<p>after a bit of shit-talking and reassurance from other sysadmins, i reexamined and realized that <a href="/~fosslinux/">~fosslinux</a> had only run <code>nmap</code> for addresses in the <code>10.0.0.0/8</code> space. the <code>10/8</code> address space is intended to not be addressable outside the local space. how could <a href="https://hetzner.com">hetzner</a> have found out about a localhost network probe!?</p>
|
|
|
|
<p>finally, after speaking with more people than i expected to speak with in one day, i ended up sending three different support emails to hetzner support, which finally resulted in them unlocking the ip.</p>
|
|
|
|
<p>it's definitely time to research redundancy options!</p>
|
|
|
|
<p>tags: <a href='tag_post-mortem.html'>post-mortem</a>, <a href='tag_linux.html'>linux</a>, <a href='tag_sysadmin.html'>sysadmin</a></p>
|
|
<!-- text end -->
|
|
<h3><a class="ablack" href="upsides-of-new-dns-nameservers.html">
|
|
upsides of new dns nameservers
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201808141505.38# -->
|
|
<div class="subtitle">August 14, 2018 —
|
|
~ben
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<ul>
|
|
<li>no more google</li>
|
|
<li>no more google</li>
|
|
<li>automated certbot validation for letsencrypt wildcard certs!! no more manual TXT records every three months!</li>
|
|
</ul>
|
|
|
|
<p>tags: <a href='tag_dns.html'>dns</a>, <a href='tag_linux.html'>linux</a>, <a href='tag_tilde.html'>tilde</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<!-- text end -->
|
|
<h3><a class="ablack" href="dns-shenanigans-post-mortem.html">
|
|
dns shenanigans post-mortem
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201808141503.49# -->
|
|
<div class="subtitle">August 14, 2018 —
|
|
~ben
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<p>let's start by saying i probably should have done a bit more research before
|
|
diving head-first into this endeavor.</p>
|
|
|
|
<p>i've been thinking about transferring my domains off google domains for some
|
|
time now, as part of my personal goal to self host and limit my dependence on
|
|
google and other large third-party monstrosities. along that line, i asked for
|
|
registrar recommendations. <a href="https://tomasino.tilde.team">~tomasino</a> responded
|
|
with <a href="https://namesilo.com">namesilo</a>. i found that they had $3.99 registrations
|
|
for .team and .zone domains, which is 1/10th the cost of the $40 registration
|
|
on google domains.</p>
|
|
|
|
<p>i started out by getting the list of domains from the google console. 2 or 3
|
|
of them had been registered within the last 60 days, so i wasn't able to
|
|
transfer those just yet. i grabbed all the domain unlock codes and dropped
|
|
them into namesilo. i failed to realize that the dns panel on google domains
|
|
would disappear as soon as it went through, but more importantly that the
|
|
nameservers would be left pointing to the old defunct google domains ones.</p>
|
|
|
|
<p>i updated the nameservers as soon as i realized this error from the namesilo
|
|
panel. some of the domains propagated quickly. others, not so much. tilde.team
|
|
was still in a state of flux between the old and new nameservers.</p>
|
|
|
|
<p>in a rush to get the dns problem fixed, and under recommendation from several
|
|
people on irc, i decided to switch the nameservers for tilde.team and tilde.zone
|
|
to cloudflare, leaving another layer of flux for the dns to be stuck in...</p>
|
|
|
|
<p>of the five domains that i moved to cloudflare, 3 returned with a dnssec error,
|
|
claiming that i needed to remove the DS record from that zone. d'oh!</p>
|
|
|
|
<p>i removed the dnssec from those affected domains, so we should be good to go
|
|
as soon as it all propagates through the fickle beast that is dns.</p>
|
|
|
|
<p>tags: <a href='tag_linux.html'>linux</a>, <a href='tag_sysadmin.html'>sysadmin</a>, <a href='tag_tilde.html'>tilde</a>, <a href='tag_dns.html'>dns</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<!-- text end -->
|
|
<h3><a class="ablack" href="dotfiles.html">
|
|
dotfiles
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201807221926.26# -->
|
|
<div class="subtitle">July 22, 2018 —
|
|
~ben
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<p>finally got around to updating my <a href="https://git.tilde.team/ben/dotfiles">dotfiles</a> to use gnu stow.
|
|
i adapted <a href="https://github.com/jamestomasino/dotfiles/blob/master/Makefile">~tomasino's makefile</a>
|
|
for use with the configs that i'm keeping with it.</p>
|
|
|
|
<p>now i just need to figure out why my ssh config doesn't copy/symlink my config to ~/.ssh when it
|
|
already exists.</p>
|
|
|
|
<p>tags: <a href='tag_linux.html'>linux</a>, <a href='tag_dotfiles.html'>dotfiles</a>, <a href='tag_git.html'>git</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<!-- text end -->
|
|
<h3><a class="ablack" href="hi-there.html">
|
|
hi there
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201710021611.13# -->
|
|
<div class="subtitle">October 02, 2017 —
|
|
tildeman
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<p>welcome to my tildeblog</p>
|
|
|
|
<p>you are here.</p>
|
|
|
|
<p>i like to mess around with linux sysadmin shtuff.
|
|
<a href="https://tilde.team">tilde.team</a> is my baby. let me know if you want to get involved.</p>
|
|
|
|
<p>thanks for being awesome. keep it up.</p>
|
|
|
|
<p>tags: <a href='tag_linux.html'>linux</a>, <a href='tag_sysadmin.html'>sysadmin</a>, <a href='tag_ubuntu.html'>ubuntu</a>, <a href='tag_tilde.html'>tilde</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<!-- text end -->
|
|
<h3><a class="ablack" href="lxd-networking-and-additional-ips.html">
|
|
lxd networking and additional IPs
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201807261534.50# -->
|
|
<div class="subtitle">July 26, 2018 —
|
|
~ben
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<p>now that tilde.team is on a fancy-shmancy new dedi server, i've tried to get a secondary IP address
|
|
assigned to a lxd container (which i plan to use for my personal stuff). lxd shows that the secondary
|
|
IP is being picked up by that container, but i'm still seeing the host machine's IP as the external
|
|
address.</p>
|
|
|
|
<p>i'm not sure how i'll need to configure the network settings on the host machine (now that we're running
|
|
ubuntu 18.04 and it uses netplan for configs and not /etc/network/interfaces). another confusing thing is
|
|
that the main config in /etc/netplan says that the network config is handled by systemd-networkd...</p>
|
|
|
|
<p>at least i have through the end of the year when my current vps runs out to get this up and running.</p>
|
|
|
|
<p>ping me on <a href="https://tilde.chat">irc</a> or <a href="mailto:ben@tilde.team">email</a> if you have experience with this.</p>
|
|
|
|
<p>tags: <a href='tag_linux.html'>linux</a>, <a href='tag_sysadmin.html'>sysadmin</a>, <a href='tag_ubuntu.html'>ubuntu</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<!-- text end -->
|
|
<h3><a class="ablack" href="no-more-google.html">
|
|
no more google
|
|
</a></h3>
|
|
<!-- bashblog_timestamp: #201808142336.05# -->
|
|
<div class="subtitle">August 14, 2018 —
|
|
~ben
|
|
</div>
|
|
<!-- text begin -->
|
|
|
|
<p>not sure if this is appropriately tagged, but i didn't feel like making a new
|
|
one.</p>
|
|
|
|
<p>i figured i should probably get some notes down about moving off google.</p>
|
|
|
|
<p>to start, i'll get a list of the things i was able to easily replace:</p>
|
|
|
|
<ul>
|
|
<li>gmail => <a href="https://tilde.team/wiki/?page=email">@tilde.team mail</a></li>
|
|
<li>google drive => <a href="https://syncthing.net">syncthing</a> (with a persistent node running on my personal vps)</li>
|
|
</ul>
|
|
|
|
<p>i'm still using:</p>
|
|
|
|
<ul>
|
|
<li>gplay music/youtube</li>
|
|
<li>google maps (open streetmap isn't good enough to replace it)</li>
|
|
<li>google photos - but this is going to be replaced long-term with syncthing</li>
|
|
</ul>
|
|
|
|
<p>tags: <a href='tag_linux.html'>linux</a>, <a href='tag_net-neutrality.html'>net-neutrality</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<!-- text end -->
|
|
</div>
|
|
<div id="footer">CC by-nc-nd <a href="https://tilde.team/~ben/">~ben</a> — <a href="mailto:ben@tilde.team">ben@tilde.team</a><br/>
|
|
generated with <a href="https://tildegit.org/team/bashblog">bashblog</a>, a single bash script to easily create blogs like this one</div>
|
|
</div></div>
|
|
<script src="https://utteranc.es/client.js"
|
|
repo="benharri/tilde"
|
|
issue-term="title"
|
|
crossorigin="anonymous"
|
|
theme="github-dark"
|
|
async>
|
|
</script>
|
|
|
|
</div>
|
|
<br>
|
|
</body></html>
|