Welcome to the PLCT Lab

We Write Compilers, Runtimes, and Emulators.

View on GitHub

Create a minimal Android system for RISC-V

Chen Wang, 2020.11.24

After a period of hard work, we can now run an Android “minimal system” on QEMU of RISC-V.

The following is a brief summary of the current work. Although the road ahead is still long, there is something to have a look.

1. Requirement analysis

The full name of our project is “AOSP for RISC-V”, and all the source code is currently opened on github: https://github.com/aosp-riscv. The ultimate goal of our project is to port Android on RISC-V. Of course, this goal is huge.

But in the short term, we still have a small goal, which is described in one sentence: based on the RISC-V platform, realize the kernel part of Android running on QEMU, and run the Android Shell.

Based on the above objectives, the specific analysis is to realize a minimal Android system. The meaning of “minimal system” here is the so-called “bootable unix-style command line operating system”. In the traditional sense, a complete “minimal system” is described on the left side of the figure below. From bottom to top, the bottom is the hardware (note: hardware is not part of our “minimal system”). The first layer of software running on the hardware is the “Operating System Kernel”, and upon the OS kernel is the “C library”. Based on the C Library, we can build a minimal file system, which is essentially a bunch of command-line tools. These command-line tools must include at least a “init”, which is used to start the basic login shell in cooperation with the kernel, and one “Shell” is used to interact with users and call other tools and programs. With this “minimal system”, our big goal has a foundation.

a minimal unix-style os

Through the investigation of AOSP, I roughly summarized the work we need to achieve as follows:

2. Introduction to porting work

The above talked about what needs to be done in the overall porting work. In fact, there are still many details in the specific implementation. Since our final goal (to transplant AOSP as a whole to RISC-V) is far from being achieved, I will briefly sort out what I have achieved so far, just for memo:

bionic libc

3. Steps to make this minimal system

After some work, the above small goal has been initially completed, and at least one of the “minimal Android systems” we defined above can be launched on QEMU. The related porting and modification have been opened on github. Allow me brief following steps and you are welcomed to have a try and test, submit PR, or directly participate in our AOSP porting work.

3.1 Environmental preparation

The experiment is based on Ubuntu 20.04 LTS

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04 LTS
Release: 20.04
Codename: focal

The software that needs to be installed in advance is as follows:

$ sudo apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev \
                  gawk build-essential bison flex texinfo gperf libtool patchutils bc \
                  zlib1g-dev libexpat-dev git \
                  libglib2.0-dev libfdt-dev libpixman-1-dev \
                  libncurses5-dev libncursesw5-dev

Then create a working directory riscv64-linux, the following operations are performed under this directory.

$ mkdir riscv64-linux
$ cd riscv64-linux

3.1.1 Building GNU toolchain

Download source code

$ git clone https://github.com/riscv/riscv-gnu-toolchain

Enter the source directory:

$ cd riscv-gnu-toolchain

Note that the main repository of clone above does not contain the contents of the sub-repositories, so you need to continue to update the sub-repositories. Note that the sub-repository of qemu is excluded first, because the complete download of qemu is too large; second, qemu is actually not needed for the toolchain compilation itself.

$ git rm qemu
$ git submodule update --init --recursive

Wait patiently for the sub-repository download to complete.

Note that due to I want to install the tools to /opt/riscv64, so sudo is required for make.

$ ./configure --prefix=/opt/riscv64
$ sudo make linux -j $(nproc)

Export the installation path of the toolchain. You can also write to the .bashrc file.

export PATH="$PATH:/opt/riscv64/bin"

Test whether the toolchain is installed successfully.

$ riscv64-unknown-linux-gnu-gcc -v

Output similar to the following shows that the toolchain is compiled and installed normally.

Using built-in specs.
COLLECT_GCC=riscv64-unknown-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/opt/riscv64/libexec/gcc/riscv64-unknown-linux-gnu/10.1.0/lto-wrapper
Target: riscv64-unknown-linux-gnu
Configured with: /home/u/ws/riscv64-linux/riscv-gnu-toolchain/riscv-gcc/configure --target=riscv64-unknown-linux-gnu --prefix=/opt/riscv64 --with-sysroot= /opt/riscv64/sysroot --with-system-zlib --enable-shared --enable-tls --enable-languages=c,c++,fortran --disable-libmudflap --disable-libssp --disable-libquadmath- -disable-libsanitizer --disable-nls --disable-bootstrap --src=.././riscv-gcc --disable-multilib --with-abi=lp64d --with-arch=rv64imafdc --with-tune =rocket'CFLAGS_FOR_TARGET=-O2 -mcmodel=medlow''CXXFLAGS_FOR_TARGET=-O2 -mcmodel=medlow'
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.1.0 (GCC)

3.1.2 Build LLVM/Clang tool chain

Make sure to return to the working directory riscv64-linux first.

After updating Ubuntu 20.04 LTS to the latest state, the software requirements for building llvm/clang should have been supported by default. Other tools needed in the compilation process are basically available on Ubuntu. If something is missed, please install it yourself.

Download the source code of llvm. The official source code repository is at github: https://github.com/llvm/llvm-project.

$ git clone https://github.com/llvm/llvm-project

After downloading, enter the root directory of the source code repository and check out the corresponding version. I choose thhe official release version 10.0.1-final and switch to the 10.x branch.

$ cd llvm-project/
$ git checkout release/10.x
$ mkdir build
$ cd build
$ cmake -G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=../install \
-DLLVM_TARGETS_TO_BUILD="RISCV" \
-DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi" \
-DLLVM_DEFAULT_TARGET_TRIPLE="riscv64-unknown-linux-gnu" \
../llvm
$ make -j $(nproc)
$ make install

Simply check the results of the installation.

$ ls ../install/ -l
total 20
drwxrwxr-x 2 u u 4096 October 9 11:37 bin
drwxrwxr-x 7 u u 4096 Oct 9 11:37 include
drwxrwxr-x 4 u u 4096 October 9 11:37 lib
drwxrwxr-x 2 u u 4096 October 9 11:37 libexec
drwxrwxr-x 7 u u 4096 October 9 11:37 share

Check the generated clang version:

$ ../install/bin/clang -v
clang version 10.0.1 (https://github.com/llvm/llvm-project.git ef32c611aa214dea855364efd7ba451ec5ec3f74)
Target: riscv64-unknown-linux-gnu
Thread model: posix
InstalledDir: ......

In order to directly enter clang on the command line to run the compiler later, add the path where the clang tool is installed to the PATH environment variable.

3.1.3 Build QEMU

Make sure to return to the working directory riscv64-linux first.

Download source code. We recommend the method of downloading the corresponding source code compression package directly according to the version number you need. This experiment uses the latest v5.1.0.

$ wget https://download.qemu.org/qemu-5.1.0.tar.xz
$ tar xvJf qemu-5.1.0.tar.xz

Note that the uppercase J is in the option when decompressing.

Compile and install. Enter the decompressed directory and perform configuration, compilation and installation. Note that I configure to install under /opt/qemu, so sudo is required for make install.

$ cd qemu-5.1.0/
$ ./configure --target-list=riscv64-softmmu --prefix=/opt/qemu
$ make -j $(nproc)
$ sudo make install

Export the installation directory of qemu to PATH and you can verify if the installation is correct:

$ qemu-system-riscv64 --version

An output similar to the following indicates that qemu is working properly:

QEMU emulator version 5.1.0
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers

3.2 Build Android kernel

Download Android kernel source code:

$ git clone https://android.googlesource.com/kernel/common
$ cd common
$ git checkout android-5.4-stable

Download Android kernel configuration. Besides the patches in the source code. Google has its own specific configuration for the Android kernel, so we need to add these configurations.

$ git clone https://android.googlesource.com/kernel/configs
$ cd configs

The key is to merge the Android configuration fragments into the default configuration for the RISC-V platform.

Assume our workspace is structured as below:

$ tree -L 2
.
├── aosp-kernel
│   ├── common
│   └── configs
└── riscv64-linux
    ├── llvm-project
    ├── qemu-5.1.0
    └── riscv-gnu-toolchain

First enter the configs repository and switch to android11-release

$ cd configs
$ git checkout android11-release

Then enter the common warehouse, clean it up first, then make sure to switch to the android-5.4-stable branch and then perform the merge configuration operation. After completion, you can compile directly. Note that the .config file will be automatically generated in the root directory of the kernel source code after running the script merge_config.sh, and I only merged the base configuration here.

$ cd common
$ make ARCH=riscv distclean
$ git checkout android-5.4-stable
$ ARCH=riscv scripts/kconfig/merge_config.sh arch/riscv/configs/defconfig ../configs/r/android-5.4/android-base.config
$ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- -j $(nproc)

3.3 Building a “Minimal System”

This part actually includes the production of the bionic libc static library plus toybox and mksh, all have been automated.

Download the source code first. After entering your working directory, execute the following command:

$ git clone https://github.com/aosp-riscv/port_bionic.git
$ cd port_bionic/
$ git checkout develop
$ git submodule init
$ git submodule update --progress

After downloading, you can make it, the steps are also very simple, run the following command in the port_bionic directory.

$ make

If you want to make an img of the root file system, you can run the mkrootfs.sh script. Note that you’d better check the mkrootfs.sh script before running, adjust it yourself and make sure that the PATH_QEMU variable points to the correct QEMU installation directory, such as in section 3.1.3 above, that is /opt/qemu/bin. Then execute mkrootfs.sh with sudo permissions.

$ sudo ./mkrootfs.sh

The content generated by the above steps is in the out directory:

$ tree -L 3 out
out
├── bin
│   ├── sh
│   ├── toybox
│   └── unstripped
│       ├── sh
│       └── toybox
├── lib
│   ├── crtbegin_static.o
│   ├── crtend.o
│   └── static
│       └── libc.a
├── obj
│   ├── xxx.o
│   ├── xxx.o.d
│   └── ......
└── rootfs.img

among them

3.4 Launch the system

Execute the following command, remember to set the path of the kernel Image and rootfs.img to the correct location on your machine. My example here is as follows:

$ qemu-system-riscv64 -M virt -m 256M -nographic -kernel ../../../aosp-kernel/common/arch/riscv/boot/Image -drive file=out/rootfs.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -append "root=/dev/vda rw console=ttyS0"

Finally, note that since the shutdown command has not yet been implemented, only Ctrl+A X can be used to poweroff the system, that is, press the Ctrl and A keys at the same time, and release and then press the X key to force QEMU to exit.

At present, the minimal system only provides very little functions. Although it is still very imperfect, it can be run after all, and it is definitely a “minimal Android system”. What’s more, it is also an Android running on the RISC-V architecture platform!