<p>we had something of an outage on november 13, 2018 on tilde.team. </p>
<p>i awoke, not suspecting anything to be amiss. as soon as i logged in to check my email and irc mentions, it became clear.</p>
<p>tilde.team was at the least inaccessible, and at the worst, down completely. according the message in my inbox, there hade been an attempted "attack" from my IP.</p>
<p>We have indications that there was an attack from your server.
Please take all necessary measures to avoid this in the future and to solve the issue.</p>
</blockquote>
<p>at this point, i have no idea what could have happened over night while i'm sleeping. the timestamp shows that it arrive only 30 minutes after i'd turned in for the night.</p>
<p>when i finally log on in the morning to check mails and irc mentions, i find that i'm unable to connect to tilde.team... strange, but ok; time to troubleshoot. i refresh the <ahref="https://mail.tilde.team">webmail</a> to see what i'm missing. it ends up failing to find the server. even stranger! i'd better get the mails off my phone if they're on my @tilde.team mail!</p>
<p>here, i launch in to full debugging mode: what command was it? who ran it? </p>
<p>search <code>~/.bash_history</code> per user was not very successful. nothing i could find was related to net or map. i had checked <code>sudo grep nmap /home/*/.bash_history</code> and many other commands. </p>
<p>at this point, i had connected with other ~teammates across other irc nets (<ahref="https://hashbang.sh/">#!</a>, <ahref="https://tilde.town">~town</a>, etc). among suggestions to check <code>/var/log/syslog</code>, <code>/var/log/kern.log</code>, and <code>dmesg</code>, i finally decided to check <code>ps</code>. <code>ps -ef | grep nmap</code> yielded nmap on an obscured uid and gid, which is shortly established to belong to a container i had provisioned for <ahref="/~fosslinux/">~fosslinux</a>.</p>
<p>i'm not considering methods of policing access to any site over port 80 and port 443. this is crazy. how do you police <code>nmap</code> when it isn't scanning on every port?</p>
<p>after a bit of shit-talking and reassurance from other sysadmins, i reexamined and realized that <ahref="/~fosslinux/">~fosslinux</a> had only run <code>nmap</code> for addresses in the <code>10.0.0.0/8</code> space. the <code>10/8</code> address space is intended to not be addressable outside the local space. how could <ahref="https://hetzner.com">hetzner</a> have found out about a localhost network probe!?</p>
<p>finally, after speaking with more people than i expected to speak with in one day, i ended up sending three different support emails to hetzner support, which finally resulted in them unlocking the ip.</p>
<p>it's definitely time to research redundancy options!</p>