# Setup: Ubuntu host, Odroid C2 board, arm64 kernel
These are the instructions on how to fuzz the kernel on an [Odroid C2](http://www.hardkernel.com/main/products/prdt_info.php) board using Ubuntu 14.04 on the host machine and Ubuntu on the Odroid.
## Hardware setup
### Required hardware
Your hardware setup must satisfy the following requirements:
1. Host machine should be able to read the Odroid kernel log.
2. Host machine should be able to ssh to the Odroid board.
3. Host machine should be able to forcefully reboot the Odroid.
The particular setup described below requires the following hardware:
1. [Odroid C2 board](http://www.hardkernel.com/main/products/prdt_info.php)
2. SD card (8 GB should be enough)
3. SD card reader (like [this one](https://www.amazon.de/gp/product/B009D79VH4/ref=oh_aui_detailpage_o06_s00?ie=UTF8&psc=1))
4. [USB-UART cable](http://www.hardkernel.com/main/products/prdt_info.php?g_code=G134111883934)
5. USB Ethernet adapter (like [this one](https://www.amazon.de/Apple-MC704LL-A-USB-Ethernet-Adapter/dp/B00W7W9FK0/ref=dp_ob_title_ce))
6. Ethernet cable
7. USB hub with [Per Port Power Switching support](http://www.gniibe.org/development/ac-power-control-by-USB-hub/index.html) (like D-Link DUB H7, **silver** edition).
8. [USB-DC Plug Cable](http://www.hardkernel.com/main/products/prdt_info.php?g_code=G141637559827)
If you decide to use a different setup, you will need to update [Odroid-related code](https://github.com/google/syzkaller/blob/master/vm/odroid/odroid.go) in syzkaller manager.
### Setup Odroid
1. Download and flash [Ubuntu image](http://odroid.com/dokuwiki/doku.php?id=en:c2_release_linux_ubuntu) onto SD card as described [here](http://odroid.com/dokuwiki/doku.php?id=en:odroid_flashing_tools).
2. Connect USB-UART cable and install minicom as described [here](http://odroid.com/dokuwiki/doku.php?id=en:usb_uart_kit).
3. Connect power plug, Odroid will start booting, make sure you see bootloader and kernel logs in minicom.
4. Make sure you can login through minicom as user `odroid` with password `odroid`. This user is a sudoer.
When `systemd` starts Odroid stops sending kernel logs to UART.
To fix this login to the Odroid board and add `kernel.printk = 7 4 1 3` line to `/etc/sysctl.conf` and then do `sysctl -p`:
``` bash
$ cat /etc/sysctl.conf | tail -n 1
kernel.printk = 7 4 1 3
$ sudo sysctl -p
kernel.printk = 7 4 1 3
```
Now make sure you can see kernel messages in minicom:
```
$ echo "Some message" | sudo tee /dev/kmsg
Some message
[ 233.128597] Some message
```
### Setup network
1. Connect USB Ethernet adapter to the host machine.
2. Use Ethernet cable to connect Odroid and the host adapter.
3. Use minicom to modify `/etc/network/interfaces` on Odroid:
```
auto eth0
iface eth0 inet static
address 172.16.0.31
gateway 172.16.0.1
netmask 255.255.255.0
```
4. Reboot Odroid.
5. Setup the interface on the host machine (though Network Manager or via `/etc/network/interfaces`):
```
auto eth1
iface eth1 inet static
address 172.16.0.30
gateway 172.16.0.1
netmask 255.255.255.0
```
6. You should now be able to ssh to Odroid (user `root`, password `odroid`):
``` bash
$ ssh root@172.16.0.31
root@172.16.0.31's password:
...
Last login: Thu Feb 11 11:30:51 2016
root@odroid64:~#
```
### Setup USB hub
To perform a hard reset of the Odroid board (by turning off power) I used a D-Link DUB H7 USB hub (**silver** edition, not the black one).
This hub has support for a feature called [Per Port Power Switching](http://www.gniibe.org/development/ac-power-control-by-USB-hub/index.html), which allows to turn off power on a selected port on the hub remotely (via USB connection to the host machine) .
[To be able to open the hub device entry](http://www.janosgyerik.com/adding-udev-rules-for-usb-debugging-android-devices/) under `/dev/` without being root, add the following file to `/etc/udev/rules.d/` on the host machine:
``` bash
$ cat /etc/udev/rules.d/10-local.rules
SUBSYSTEM=="usb", ATTR{idVendor}=="2001", ATTR{idProduct}=="f103", MODE="0664", GROUP="plugdev"
```
`idVendor` and `idProduct` should correspond to the hub vendor and product id (can be seen via `lsusb`).
Don't forget to replug the hub after you add this file.
``` bash
$ lsusb
...
Bus 003 Device 026: ID 2001:f103 D-Link Corp. DUB-H7 7-port USB 2.0 hub
...
```
Communication with the hub is done by sending USB control messages, which requires `libusb`:
``` bash
sudo apt-get install libusb-dev libusb-1.0-0-dev
```
Now plug in the hub and try to switch power on some of it's ports.
For that you can use the [hub-ctrl.c](https://github.com/codazoda/hub-ctrl.c) tool by Niibe Yutaka or it's [ simplified Go analog](https://gist.github.com/xairy/37264952ff35da6e7dcf51ef486368e5):
``` bash
$ go run hub.go -bus=3 -device=26 -port=6 -power=0
Power turned off on port 6
$ go run hub.go -bus=3 -device=26 -port=6 -power=1
Power turned on on port 6
```
Note, that the DUB-H7 hub has a weird port numbering: `5, 6, 1, 2, 7, 3, 4` from left to right.
Connect the Odroid board with a power plug to one of the USB hub ports and make sure you can forcefully reboot the Odroid by turning the power off and back on on this port.
## Cross-compiler
You need to compile full GCC cross-compiler tool-chain for aarch64 as described [here](http://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/) (including the standard libraries).
Use GCC revision 242378 (newer revisions should work as well, but weren't tested).
The result should be a `$PREFIX` directory with cross-compiler, standard library headers, etc.
```
$ ls $PREFIX
aarch64-linux bin include lib libexec share
```
## Kernel
Set environment variables, they will be detected and used during kernel compilation:
``` bash
export PATH="$PREFIX/bin:$PATH"
export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-
```
Clone the linux-next kernel into `$KERNEL`:
``` bash
git clone https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git $KERNEL
cd $KERNEL
```
Apply the following patch, otherwise building the kernel with newer GCC fails (the patch is taken from [here](https://patchwork.kernel.org/patch/9380181/)):
``` makefile
diff --git a/Makefile b/Makefile
index 165cf9783a5d..ff8b40dca9e2 100644
--- a/Makefile
+++ b/Makefile
@@ -653,6 +653,11 @@ KBUILD_CFLAGS += $(call cc-ifversion, -lt, 0409, \
# Tell gcc to never replace conditional load with a non-conditional one
KBUILD_CFLAGS += $(call cc-option,--param=allow-store-data-races=0)
+# Stop gcc from converting switches into a form that defeats dead code
+# elimination and can subsequently lead to calls to intentionally
+# undefined functions appearing in the final link.
+KBUILD_CFLAGS += $(call cc-option,--param=max-fsm-thread-path-insns=1)
+
include scripts/Makefile.gcc-plugins
ifdef CONFIG_READABLE_ASM
```
Apply the following patch to disable KASAN bug detection on stack and globals (kernel doesn't boot, KASAN needs to be fixed):
``` makefile
diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan
index 9576775a86f6..8bc4eb36fc1b 100644
--- a/scripts/Makefile.kasan
+++ b/scripts/Makefile.kasan
@@ -11,7 +11,6 @@ CFLAGS_KASAN_MINIMAL := -fsanitize=kernel-address
CFLAGS_KASAN := $(call cc-option, -fsanitize=kernel-address \
-fasan-shadow-offset=$(KASAN_SHADOW_OFFSET) \
- --param asan-stack=1 --param asan-globals=1 \
--param asan-instrumentation-with-call-threshold=$(call_threshold))
ifeq ($(call cc-option, $(CFLAGS_KASAN_MINIMAL) -Werror),)
```
Configure the kernel (you might wan't to enable more configs as listed [here](kernel_configs.md)):
``` bash
make defconfig
# Edit .config to enable the following configs:
# CONFIG_KCOV=y
# CONFIG_KASAN=y
# CONFIG_KASAN_INLINE=y
# CONFIG_TEST_KASAN=m
# CONFIG_PANIC_ON_OOPS=y
make oldconfig
```
Build the kernel:
``` bash
make -j48 dtbs Image modules LOCALVERSION=-xc2
```
## Installation
Install the `mkimage` util with arm64 support (part of the `u-boot-tools` package).
You might have it by default, but it's not available on Ubuntu 14.04 in the default package repos.
In this case download the package from [here](https://launchpad.net/ubuntu/xenial/amd64/u-boot-tools/2016.01+dfsg1-2ubuntu1) and use `sudo dpkg -i` to install.
Insert the SD card reader with the SD card inside into the host machine.
You should see two partitions automounted (or mount them manually), for example `sdb1` mounted at `$MOUNT_PATH/boot` and `sdb2` mounted at `$MOUNT_PATH/rootfs`.
Build the kernel image:
``` bash
mkimage -A arm64 -O linux -T kernel -C none -a 0x1080000 -e 0x1080000 -n linux-next -d arch/arm64/boot/Image ./uImage
```
Copy the kernel image, modules and device tree:
``` bash
KERNEL_VERSION=`cat ./include/config/kernel.release`
cp ./uImage $MOUNT_PATH/boot/uImage-$KERNEL_VERSION
make modules_install LOCALVERSION=-xc2 INSTALL_MOD_PATH=$MOUNT_PATH/rootfs/
cp ./arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dtb $MOUNT_PATH/boot/meson-gxbb-odroidc2-$KERNEL_VERSION.dtb
cp .config $MOUNT_PATH/boot/config-$KERNEL_VERSION
```
Backup the old bootloader config; if something doesn't work with the new kernel, you can always roll back to the old one by restoring `boot.ini`:
``` bash
cd $MOUNT_PATH/boot/
cp boot.ini boot.ini.orig
```
Replace the bootloader config `boot.ini` (based on the one taken from [here](http://forum.odroid.com/viewtopic.php?p=162045#p162045)) with the following; don't forget to update `version`:
```
ODROIDC2-UBOOT-CONFIG
# Set version to $KERNEL_VERSION
setenv version 4.11.0-rc1-next-20170308-xc2-dirty
setenv uImage uImage-${version}
setenv fdtbin meson-gxbb-odroidc2-${version}.dtb
setenv initrd_high 0xffffffff
setenv fdt_high 0xffffffff
setenv uimage_addr_r 0x01080000
setenv fdtbin_addr_r 0x01000000
# You might need to use root=/dev/mmcblk0p2 below, try booting and see if the current one works.
setenv bootargs "console=ttyAML0,115200 root=/dev/mmcblk1p2 rootwait ro fsck.mode=force fsck.repair=yes net.ifnames=0 oops=panic panic_on_warn=1 panic=86400 systemd.show_status=no"
fatload mmc 0:1 ${fdtbin_addr_r} ${fdtbin}
fatload mmc 0:1 ${uimage_addr_r} ${uImage}
bootm ${uimage_addr_r} - ${fdtbin_addr_r}
```
Sync and unmount:
``` bash
sync
umount $MOUNT_PATH/boot
umount $MOUNT_PATH/rootfs
```
Now plug the SD card into the Odroid board and boot.
The new kernel should now be used.
It makes sense to ensure that you still can ssh to Odroid.
## Syzkaller
Generate ssh key and copy it to Odroid:
``` bash
mkdir ssh
ssh-keygen -f ssh/id_rsa -t rsa -N ''
ssh root@172.16.0.31 "mkdir /root/.ssh/"
scp ./ssh/id_rsa.pub root@172.16.0.31:/root/.ssh/authorized_keys
```
Now make sure you can ssh with the key:
``` bash
ssh -i ./ssh/id_rsa root@172.16.0.31
```
Build syzkaller with `odroid` build tag:
``` bash
make GOTAGS=odroid TARGETARCH=arm64
```
Use the following config:
```
{
"target": "linux/arm64",
"http": "127.0.0.1:56741",
"workdir": "/syzkaller/workdir",
"kernel_obj": "/linux-next",
"syzkaller": "/go/src/github.com/google/syzkaller",
"sshkey": "/odroid/ssh/id_rsa",
"rpc": "172.16.0.30:0",
"sandbox": "namespace",
"reproduce": false,
"procs": 8,
"type": "odroid",
"vm": {
"host_addr": "172.16.0.30",
"slave_addr": "172.16.0.31",
"console": "/dev/ttyUSB0",
"hub_bus": 3,
"hub_device": 26,
"hub_port": 5
}
}
```
Don't forget to update:
- `workdir` (path to the workdir)
- `kernel_obj` (path to kernel build directory)
- `sshkey` (path to the generated ssh private key)
- `vm.console` (serial device you used in `minicom`)
- `vm.hub_bus` (number of the bus to which USB hub is connected, view with `lsusb`)
- `vm.hub_device` (device number for the USB hub, view with `lsusb`)
- `vm.hub_port` (number of the USB hub port to which Odroid power plug is connected)
Now start syzkaller:
``` bash
./bin/syz-manager -config=odroid.cfg
```
If you get issues after `syz-manager` starts, consider running it with the `-debug` flag.
Also see [this page](/docs/troubleshooting.md) for troubleshooting tips.