diff options
author | Matthias Maennich <maennich@google.com> | 2019-02-01 22:20:44 +0000 |
---|---|---|
committer | Matthias Maennich <maennich@google.com> | 2019-02-01 22:43:20 +0000 |
commit | 6652d74e4b3fa4175ab49f9eebb2113faa0ec29c (patch) | |
tree | 0c5cf7c2850ebed9d3c41b526ac11037ccf4fdca | |
parent | 12bf50c84f5885d9a6fba1541f7e03aedbd34df3 (diff) |
initial release of kernel/build in AOSP
Publish build scripts to build Android Kernels in AOSP. They include the
actual build script build/build.sh, helper scripts to run the
kernel/tests as well as common utilities for validating patches.
Refer to build.sh for documentation on how to invoke the kernel build.
Bug: 120282827
Signed-off-by: Matthias Maennich <maennich@google.com>
Change-Id: Id860c320ced4e9a2fd5e32b4730c7b86ac843294
-rw-r--r-- | LICENSE | 203 | ||||
-rw-r--r-- | METADATA | 17 | ||||
-rw-r--r-- | MODULE_LICENSE_APACHE2 | 0 | ||||
l--------- | NOTICE | 1 | ||||
l--------- | build.config | 1 | ||||
-rw-r--r-- | build.config.net_test | 9 | ||||
-rwxr-xr-x | build.sh | 320 | ||||
-rwxr-xr-x | build_test.sh | 60 | ||||
-rw-r--r-- | envsetup.sh | 74 | ||||
-rwxr-xr-x | static_analysis/checkpatch.sh | 130 | ||||
-rw-r--r-- | static_analysis/checkpatch_blacklist | 128 | ||||
-rwxr-xr-x | static_analysis/checkpatch_presubmit.sh | 90 |
12 files changed, 1033 insertions, 0 deletions
@@ -0,0 +1,203 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + diff --git a/METADATA b/METADATA new file mode 100644 index 0000000..5a16d1d --- /dev/null +++ b/METADATA @@ -0,0 +1,17 @@ +name: "kernel/build" +description: + "Build scripts to build Android Kernels." + +third_party { + url { + type: HOMEPAGE + value: "https://source.android.com/setup/build/building-kernels" + } + url { + type: GIT + value: "https://android.googlesource.com/kernel/build" + } + version: "" + last_upgrade_date { year: 2019 month: 2 day: 1 } + license_type: NOTICE +} diff --git a/MODULE_LICENSE_APACHE2 b/MODULE_LICENSE_APACHE2 new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/MODULE_LICENSE_APACHE2 @@ -0,0 +1 @@ +LICENSE
\ No newline at end of file diff --git a/build.config b/build.config new file mode 120000 index 0000000..7c5f637 --- /dev/null +++ b/build.config @@ -0,0 +1 @@ +../build.config
\ No newline at end of file diff --git a/build.config.net_test b/build.config.net_test new file mode 100644 index 0000000..721dac0 --- /dev/null +++ b/build.config.net_test @@ -0,0 +1,9 @@ +ARCH=um +SUBARCH=x86_64 +CROSS_COMPILE= +EXTRA_CMDS='' +KERNEL_DIR=$(. $(dirname $0)/build.config && echo $KERNEL_DIR || echo common) +LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN= +FILES=" +linux +" diff --git a/build.sh b/build.sh new file mode 100755 index 0000000..cc2aa82 --- /dev/null +++ b/build.sh @@ -0,0 +1,320 @@ +#!/bin/bash + +# Copyright (C) 2019 The Android Open Source Project +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Usage: +# build/build.sh <make options>* +# or: +# OUT_DIR=<out dir> DIST_DIR=<dist dir> build/build.sh <make options>* +# +# Example: +# OUT_DIR=output DIST_DIR=dist build/build.sh -j24 +# +# +# The following environment variables are considered during execution: +# +# BUILD_CONFIG +# Build config file to initialize the build environment from. The location +# is to be defined relative to the repo root directory. +# Defaults to 'build.config'. +# +# OUT_DIR +# Base output directory for the kernel build. +# Defaults to <REPO_ROOT>/out/<BRANCH>. +# +# DIST_DIR +# Base output directory for the kernel distribution. +# Defaults to <OUT_DIR>/dist +# +# EXT_MODULES +# Space separated list of external kernel modules to be build. +# +# UNSTRIPPED_MODULES +# Space separated list of modules to be copied to <DIST_DIR>/unstripped +# for debugging purposes. +# +# CC +# Override compiler to be used. (e.g. CC=clang) +# +# LD +# Override linker (flags) to be used. +# +# Environment variables to influence the stages of the kernel build. +# +# SKIP_MRPROPER +# if defined, skip `make mrproper` +# +# SKIP_DEFCONFIG +# if defined, skip `make defconfig` +# +# POST_DEFCONFIG_CMDS +# Command evaluated after `make defconfig` and before `make`. +# +# IN_KERNEL_MODULES +# if defined, install kernel modules +# +# SKIP_EXT_MODULES +# if defined, skip building and installing of external modules +# +# EXTRA_CMDS +# Command evaluated after building and installing kernel and modules. +# +# SKIP_CP_KERNEL_HDR +# if defined, skip installing kernel headers. +# +# Note: For historic reasons, internally, OUT_DIR will be copied into +# COMMON_OUT_DIR, and OUT_DIR will be then set to +# ${COMMON_OUT_DIR}/${KERNEL_DIR}. This has been done to accommodate existing +# build.config files that expect ${OUT_DIR} to point to the output directory of +# the kernel build. +# +# The kernel is built in ${COMMON_OUT_DIR}/${KERNEL_DIR}. +# Out-of-tree modules are built in ${COMMON_OUT_DIR}/${EXT_MOD} where +# ${EXT_MOD} is the path to the module source code. + +set -e + +# rel_path <to> <from> +# Generate relative directory path to reach directory <to> from <from> +function rel_path() { + local to=$1 + local from=$2 + local path= + local stem= + local prevstem= + [ -n "$to" ] || return 1 + [ -n "$from" ] || return 1 + to=$(readlink -e "$to") + from=$(readlink -e "$from") + [ -n "$to" ] || return 1 + [ -n "$from" ] || return 1 + stem=${from}/ + while [ "${to#$stem}" == "${to}" -a "${stem}" != "${prevstem}" ]; do + prevstem=$stem + stem=$(readlink -e "${stem}/..") + [ "${stem%/}" == "${stem}" ] && stem=${stem}/ + path=${path}../ + done + echo ${path}${to#$stem} +} + +export ROOT_DIR=$(readlink -f $(dirname $0)/..) + +# For module file Signing with the kernel (if needed) +FILE_SIGN_BIN=scripts/sign-file +SIGN_SEC=certs/signing_key.pem +SIGN_CERT=certs/signing_key.x509 +SIGN_ALGO=sha512 + +source "${ROOT_DIR}/build/envsetup.sh" + +export MAKE_ARGS=$@ +export COMMON_OUT_DIR=$(readlink -m ${OUT_DIR:-${ROOT_DIR}/out/${BRANCH}}) +export OUT_DIR=$(readlink -m ${COMMON_OUT_DIR}/${KERNEL_DIR}) +export MODULES_STAGING_DIR=$(readlink -m ${COMMON_OUT_DIR}/staging) +export MODULES_PRIVATE_DIR=$(readlink -m ${COMMON_OUT_DIR}/private) +export DIST_DIR=$(readlink -m ${DIST_DIR:-${COMMON_OUT_DIR}/dist}) +export UNSTRIPPED_DIR=${DIST_DIR}/unstripped +export KERNEL_UAPI_HEADERS_DIR=$(readlink -m ${COMMON_OUT_DIR}/kernel_uapi_headers) + +cd ${ROOT_DIR} + +export CLANG_TRIPLE CROSS_COMPILE CROSS_COMPILE_ARM32 ARCH SUBARCH + +mkdir -p ${OUT_DIR} +echo "========================================================" +echo " Setting up for build" +if [ -z "${SKIP_MRPROPER}" ] ; then + set -x + (cd ${KERNEL_DIR} && make O=${OUT_DIR} mrproper) + set +x +fi + +if [ -z "${SKIP_DEFCONFIG}" ] ; then +set -x +(cd ${KERNEL_DIR} && make O=${OUT_DIR} ${DEFCONFIG}) +set +x + +if [ "${POST_DEFCONFIG_CMDS}" != "" ]; then + echo "========================================================" + echo " Running pre-make command(s):" + set -x + eval ${POST_DEFCONFIG_CMDS} + set +x +fi +fi + +echo "========================================================" +echo " Building kernel" + +if [ -n "${CC}" ]; then + CC_ARG="CC=${CC}" +fi + +if [ -n "${LD}" ]; then + LD_ARG="LD=${LD}" +fi + +set -x +(cd ${OUT_DIR} && \ + make O=${OUT_DIR} ${CC_ARG} ${LD_ARG} -j$(nproc) $@) +set +x + +rm -rf ${MODULES_STAGING_DIR} +mkdir -p ${MODULES_STAGING_DIR} + +if [ -n "${IN_KERNEL_MODULES}" ]; then + echo "========================================================" + echo " Installing kernel modules into staging directory" + + (cd ${OUT_DIR} && \ + make O=${OUT_DIR} ${CC_ARG} ${LD_ARG} INSTALL_MOD_STRIP=1 \ + INSTALL_MOD_PATH=${MODULES_STAGING_DIR} modules_install) +fi + +if [[ -z "${SKIP_EXT_MODULES}" ]] && [[ "${EXT_MODULES}" != "" ]]; then + echo "========================================================" + echo " Building external modules and installing them into staging directory" + + for EXT_MOD in ${EXT_MODULES}; do + # The path that we pass in via the variable M needs to be a relative path + # relative to the kernel source directory. The source files will then be + # looked for in ${KERNEL_DIR}/${EXT_MOD_REL} and the object files (i.e. .o + # and .ko) files will be stored in ${OUT_DIR}/${EXT_MOD_REL}. If we + # instead set M to an absolute path, then object (i.e. .o and .ko) files + # are stored in the module source directory which is not what we want. + EXT_MOD_REL=$(rel_path ${ROOT_DIR}/${EXT_MOD} ${KERNEL_DIR}) + # The output directory must exist before we invoke make. Otherwise, the + # build system behaves horribly wrong. + mkdir -p ${OUT_DIR}/${EXT_MOD_REL} + set -x + make -C ${EXT_MOD} M=${EXT_MOD_REL} KERNEL_SRC=${ROOT_DIR}/${KERNEL_DIR} \ + O=${OUT_DIR} ${CC_ARG} ${LD_ARG} -j$(nproc) "$@" + make -C ${EXT_MOD} M=${EXT_MOD_REL} KERNEL_SRC=${ROOT_DIR}/${KERNEL_DIR} \ + O=${OUT_DIR} ${CC_ARG} ${LD_ARG} INSTALL_MOD_STRIP=1 \ + INSTALL_MOD_PATH=${MODULES_STAGING_DIR} modules_install + set +x + done + +fi + +if [ "${EXTRA_CMDS}" != "" ]; then + echo "========================================================" + echo " Running extra build command(s):" + set -x + eval ${EXTRA_CMDS} + set +x +fi + +OVERLAYS_OUT="" +for ODM_DIR in ${ODM_DIRS}; do + OVERLAY_DIR=${ROOT_DIR}/device/${ODM_DIR}/overlays + + if [ -d ${OVERLAY_DIR} ]; then + OVERLAY_OUT_DIR=${OUT_DIR}/overlays/${ODM_DIR} + mkdir -p ${OVERLAY_OUT_DIR} + make -C ${OVERLAY_DIR} DTC=${OUT_DIR}/scripts/dtc/dtc OUT_DIR=${OVERLAY_OUT_DIR} + OVERLAYS=$(find ${OVERLAY_OUT_DIR} -name "*.dtbo") + OVERLAYS_OUT="$OVERLAYS_OUT $OVERLAYS" + fi +done + +mkdir -p ${DIST_DIR} +echo "========================================================" +echo " Copying files" +for FILE in ${FILES}; do + if [ -f ${OUT_DIR}/${FILE} ]; then + echo " $FILE" + cp -p ${OUT_DIR}/${FILE} ${DIST_DIR}/ + else + echo " $FILE does not exist, skipping" + fi +done + +for FILE in ${OVERLAYS_OUT}; do + OVERLAY_DIST_DIR=${DIST_DIR}/$(dirname ${FILE#${OUT_DIR}/overlays/}) + echo " ${FILE#${OUT_DIR}/}" + mkdir -p ${OVERLAY_DIST_DIR} + cp ${FILE} ${OVERLAY_DIST_DIR}/ +done + +MODULES=$(find ${MODULES_STAGING_DIR} -type f -name "*.ko") +if [ -n "${MODULES}" ]; then + echo "========================================================" + echo " Copying modules files" + if [ -n "${IN_KERNEL_MODULES}" -o "${EXT_MODULES}" != "" ]; then + for FILE in ${MODULES}; do + echo " ${FILE#${MODULES_STAGING_DIR}/}" + cp -p ${FILE} ${DIST_DIR} + done + fi +fi + +if [ "${UNSTRIPPED_MODULES}" != "" ]; then + echo "========================================================" + echo " Copying unstripped module files for debugging purposes (not loaded on device)" + mkdir -p ${UNSTRIPPED_DIR} + for MODULE in ${UNSTRIPPED_MODULES}; do + find ${MODULES_PRIVATE_DIR} -name ${MODULE} -exec cp {} ${UNSTRIPPED_DIR} \; + done +fi + +if [ -z "${SKIP_CP_KERNEL_HDR}" ]; then + echo "========================================================" + echo " Installing UAPI kernel headers:" + mkdir -p "${KERNEL_UAPI_HEADERS_DIR}/usr" + make -C ${OUT_DIR} O=${OUT_DIR} ${CC_ARG} INSTALL_HDR_PATH="${KERNEL_UAPI_HEADERS_DIR}/usr" -j$(nproc) headers_install + # The kernel makefiles create files named ..install.cmd and .install which + # are only side products. We don't want those. Let's delete them. + find ${KERNEL_UAPI_HEADERS_DIR} \( -name ..install.cmd -o -name .install \) -exec rm '{}' + + KERNEL_UAPI_HEADERS_TAR=${DIST_DIR}/kernel-uapi-headers.tar.gz + echo " Copying kernel UAPI headers to ${KERNEL_UAPI_HEADERS_TAR}" + tar -czf ${KERNEL_UAPI_HEADERS_TAR} --directory=${KERNEL_UAPI_HEADERS_DIR} usr/ +fi + +if [ -z "${SKIP_CP_KERNEL_HDR}" ] ; then + echo "========================================================" + KERNEL_HEADERS_TAR=${DIST_DIR}/kernel-headers.tar.gz + echo " Copying kernel headers to ${KERNEL_HEADERS_TAR}" + TMP_DIR="/tmp" + TMP_KERNEL_HEADERS_CHILD="kernel-headers" + TMP_KERNEL_HEADERS_DIR=$TMP_DIR/$TMP_KERNEL_HEADERS_CHILD + CURDIR=$(pwd) + mkdir -p $TMP_KERNEL_HEADERS_DIR + cd $ROOT_DIR/$KERNEL_DIR; find arch -name *.h -exec cp --parents {} $TMP_KERNEL_HEADERS_DIR \; + cd $ROOT_DIR/$KERNEL_DIR; find include -name *.h -exec cp --parents {} $TMP_KERNEL_HEADERS_DIR \; + cd $OUT_DIR; find -name *.h -exec cp --parents {} $TMP_KERNEL_HEADERS_DIR \; + tar -czvf $KERNEL_HEADERS_TAR --directory=$TMP_DIR $TMP_KERNEL_HEADERS_CHILD > /dev/null 2>&1 + rm -rf $TMP_KERNEL_HEADERS_DIR + cd $CURDIR +fi + +echo "========================================================" +echo " Files copied to ${DIST_DIR}" + +# No trace_printk use on build server build +if readelf -a ${DIST_DIR}/vmlinux 2>&1 | grep -q trace_printk_fmt; then + echo "========================================================" + echo "WARN: Found trace_printk usage in vmlinux." + echo "" + echo "trace_printk will cause trace_printk_init_buffers executed in kernel" + echo "start, which will increase memory and lead warning shown during boot." + echo "We should not carry trace_printk in production kernel." + echo "" + if [ ! -z "${STOP_SHIP_TRACEPRINTK}" ]; then + echo "ERROR: stop ship on trace_printk usage." 1>&2 + exit 1 + fi +fi diff --git a/build_test.sh b/build_test.sh new file mode 100755 index 0000000..268af0b --- /dev/null +++ b/build_test.sh @@ -0,0 +1,60 @@ +#!/bin/bash + +# Copyright (C) 2019 The Android Open Source Project +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Usage: +# build/build_test.sh + +export MAKE_ARGS=$@ +export ROOT_DIR=$(dirname $(readlink -f $0)) +export NET_TEST=${ROOT_DIR}/../kernel/tests/net/test + +# if device has its own build.config.net_test in the +# root (via manifest copy rule) then use it, otherwise +# use the default one in the build/ directory. if the +# BUILD_CONFIG is already specified in the environment, +# it overrides everything (unless it does not exist.) +if [ -z "$BUILD_CONFIG" ]; then + BUILD_CONFIG=build.config.net_test +fi +if [ ! -e $BUILD_CONFIG ]; then + BUILD_CONFIG=build/${BUILD_CONFIG} +fi +export BUILD_CONFIG + +test=all_tests.sh +set -e +source ${ROOT_DIR}/envsetup.sh +export OUT_DIR=$(readlink -m ${OUT_DIR:-${ROOT_DIR}/out/${BRANCH}}) +mkdir -p ${OUT_DIR} + +# build.config.net_test sets KERNEL_DIR to "private/*", which doesn't work for +# common kernels, where the code is in "common/". Check for that here. We could +# also require that each of these kernels have their own build.config.net_test, +# but that complicates the manifests. +if ! [ -f $KERNEL_DIR/Makefile ] && [ -f common/Makefile ]; then + KERNEL_DIR=common +fi +export KERNEL_DIR=$(readlink -m ${KERNEL_DIR}) + +echo "========================================================" +echo " Building kernel and running tests " +echo " Using KERNEL_DIR: " ${KERNEL_DIR} +echo " Using OUT_DIR : " ${OUT_DIR} + +cd ${OUT_DIR} +$NET_TEST/run_net_test.sh --builder $test +echo $? +echo "======Finished running tests======" diff --git a/envsetup.sh b/envsetup.sh new file mode 100644 index 0000000..d8a9729 --- /dev/null +++ b/envsetup.sh @@ -0,0 +1,74 @@ +# source this file. Don't run it. + +# Copyright (C) 2019 The Android Open Source Project +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Usage: +# source build/envsetup.sh +# to setup your path and cross compiler so that a kernel build command is +# just: +# make -j24 + + +# TODO: Use a $(gettop) style method. +export ROOT_DIR=$PWD + +export BUILD_CONFIG=${BUILD_CONFIG:-build.config} +. ${ROOT_DIR}/${BUILD_CONFIG} + +echo "========================================================" +echo "= build config: ${ROOT_DIR}/${BUILD_CONFIG}" +cat ${ROOT_DIR}/${BUILD_CONFIG} + +# List of prebuilt directories shell variables to incorporate into PATH +PREBUILTS_PATHS=" +LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN +LINUX_GCC_CROSS_COMPILE_ARM32_PREBUILTS_BIN +CLANG_PREBUILT_BIN +LZ4_PREBUILTS_BIN +DTC_PREBUILTS_BIN +LIBUFDT_PREBUILTS_BIN +" + +for PREBUILT_BIN in ${PREBUILTS_PATHS}; do + PREBUILT_BIN=\${${PREBUILT_BIN}} + eval PREBUILT_BIN="${PREBUILT_BIN}" + if [ -n "${PREBUILT_BIN}" ]; then + # Mitigate dup paths + PATH=${PATH//"${ROOT_DIR}/${PREBUILT_BIN}:"} + PATH=${ROOT_DIR}/${PREBUILT_BIN}:${PATH} + fi +done +export PATH + +echo +echo "PATH=${PATH}" +echo + +export $(sed -n -e 's/\([^=]\)=.*/\1/p' ${ROOT_DIR}/${BUILD_CONFIG}) + +# verifies that defconfig matches the DEFCONFIG +function check_defconfig() { + (cd ${OUT_DIR} && \ + make O=${OUT_DIR} savedefconfig) + [ "$ARCH" = "x86_64" -o "$ARCH" = "i386" ] && local ARCH=x86 + echo Verifying that savedefconfig matches ${KERNEL_DIR}/arch/${ARCH}/configs/${DEFCONFIG} + RES=0 + diff ${OUT_DIR}/defconfig ${KERNEL_DIR}/arch/${ARCH}/configs/${DEFCONFIG} || + RES=$? + if [ ${RES} -ne 0 ]; then + echo ERROR: savedefconfig does not match ${KERNEL_DIR}/arch/${ARCH}/configs/${DEFCONFIG} + fi + return ${RES} +} diff --git a/static_analysis/checkpatch.sh b/static_analysis/checkpatch.sh new file mode 100755 index 0000000..4857d26 --- /dev/null +++ b/static_analysis/checkpatch.sh @@ -0,0 +1,130 @@ +#!/bin/bash + +# Copyright (C) 2019 The Android Open Source Project +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Wrapper around checkpatch.pl to filter results. + +set -e + +export STATIC_ANALYSIS_SRC_DIR=$(dirname $(readlink -f $0)) + +ROOT_DIR="${STATIC_ANALYSIS_SRC_DIR}/../../" +pushd ${ROOT_DIR} +source ${STATIC_ANALYSIS_SRC_DIR}/../envsetup.sh +export OUT_DIR=$(readlink -m ${OUT_DIR:-${ROOT_DIR}/out/${BRANCH}}) +export DIST_DIR=$(readlink -m ${DIST_DIR:-${OUT_DIR}/dist}) +mkdir -p ${DIST_DIR} + +export KERNEL_DIR=$(readlink -m ${KERNEL_DIR}) + +CHECKPATCH_PL_PATH="${KERNEL_DIR}/scripts/checkpatch.pl" +GIT_SHA1="HEAD" +PATCH_DIR="${OUT_DIR}/checkpatch/patches" +BLACKLIST_FILE="${STATIC_ANALYSIS_SRC_DIR}/checkpatch_blacklist" +RESULTS_PATH=${DIST_DIR}/checkpatch.log +RETURN_CODE=0 + +echoerr() { + echo "$@" 1>&2; +} + +# Parse flags. +CHECKPATCH_ARGS=(--show-types) +while [[ $# -gt 0 ]]; do + next="$1" + case ${next} in + --git_sha1) + GIT_SHA1="$2" + shift + ;; + --blacklisted_checks) + BLACKLIST_FILE="$2" + shift + ;; + --help) + echo "Gets a patch from git, passes it checkpatch.pl, and then reports" + echo "the subset of violations we choose to enforce." + echo "" + echo "Usage: $0" + echo " <--git_sha1 nnn> (Defaults to HEAD)" + echo " <--blacklisted_checks path_to_file> (Defaults to checkpatch_blacklist)" + echo " <args for checkpatch.pl>" + exit 0 + ;; + *) + CHECKPATCH_ARGS+=("$1") + ;; + esac + shift +done + + +# Clean up from any previous run. +if [[ -d "${PATCH_DIR}" ]]; then + rm -fr "${PATCH_DIR}" +fi +mkdir -p "${PATCH_DIR}" + +# Update blacklist. +if [[ -f "${BLACKLIST_FILE}" ]]; then + IGNORED_ERRORS=$(grep -v '^#' ${BLACKLIST_FILE} | paste -s -d,) + if [[ -n "${IGNORED_ERRORS}" ]]; then + CHECKPATCH_ARGS+=(--ignore) + CHECKPATCH_ARGS+=("${IGNORED_ERRORS}") + fi +fi + +echo "========================================================" +echo " Running static analysis..." +echo "========================================================" +echo "Using KERNEL_DIR: ${KERNEL_DIR}" +echo "Using --git_sha1: ${GIT_SHA1}" + +# Generate patch file from git. +cd ${KERNEL_DIR} +git format-patch --quiet -o "${PATCH_DIR}" "${GIT_SHA1}^1..${GIT_SHA1}" +PATCH_FILE="${PATCH_DIR}/*.patch" + +# Delay exit on non-zero checkpatch.pl return code so we can finish logging. + +# Note, it's tricky to ignore this exit code completely and instead return only +# based on the log values. For example, if the log is not empty, but contains no +# ERRORS, how do we reliabliy distinguish WARNINGS that were not blacklisted +# (or other conditions we want to ignore), from legitimate errors running the +# script itself (e.g. bad flags)? checkpatch.pl will return 1 in both cases. +# For now, include all known warnings in the blacklist, and forward this code +# unconditionally. + +set +e +"${CHECKPATCH_PL_PATH}" ${CHECKPATCH_ARGS[*]} $PATCH_FILE > "${RESULTS_PATH}" +CHECKPATCH_RC=$? +set -e + +# Summarize errors in the build log (full copy included in dist dir). +if [[ $CHECKPATCH_RC -ne 0 ]]; then + echoerr "Errors were reported from checkpatch.pl." + echoerr "" + echoerr "Summary:" + echoerr "" + { grep -r -h -E -A1 "^ERROR:" "${RESULTS_PATH}" 1>&2; } || true + echoerr "" + echoerr "See $(basename ${RESULTS_PATH}) for complete output." +fi + +echo "========================================================" +echo "Finished running static analysis." +echo "========================================================" +popd +exit ${CHECKPATCH_RC} diff --git a/static_analysis/checkpatch_blacklist b/static_analysis/checkpatch_blacklist new file mode 100644 index 0000000..b28c845 --- /dev/null +++ b/static_analysis/checkpatch_blacklist @@ -0,0 +1,128 @@ +# Lists errors that should be ignored. +# To see the type of an error, run checkpatch with --show_types. +GERRIT_CHANGE_ID +# All WARNINGS are ignored +# TODO(mikewallstedt): Consider modifying checkpatch.pl such that we can +# ignore by level (this is non-trivial now because we have dozens of versions). +ALLOC_ARRAY_ARGS +ALLOC_WITH_MULTIPLY +ARM_BARRIER +ARRAY_SIZE +AVOID_EXTERNS +BAD_AUTHOR +BAD_SIGN_OFF +BLOCK_COMMENT_STYLE +BRACES +COMMIT_LOG_LONG_LINE +CONFIG_DESCRIPTION +CONFIG_EXPERIMENTAL +CONFIG_TYPE_BOOLEAN +CONSIDER_COMPLETION +CONSIDER_KSTRTO +CONSTANT_COMPARISON +CONSTANT_CONVERSION +CONST_CONST +CONST_STRUCT +CVS_KEYWORD +DEEP_INDENTATION +DEFAULT_NO_BREAK +DEFINE_PCI_DEVICE_TABLE +DEPRECATED_VARIABLE +DEPRECATED_VREG_APIS +DO_WHILE_MACRO_WITH_TRAILING_SEMICOLON +EMAIL_SUBJECT +ENOSYS +EXPEDITED_RCU_GRACE_PERIOD +EXPORTED_WORLD_WRITABLE +EXPORT_SYMBOL +FILE_PATH_CHANGES +FUNCTION_ARGUMENTS +GCC_BINARY_CONSTANT +GPIOMUX_IN_BOARD +HEXADECIMAL_BOOLEAN_TEST +HOTPLUG_SECTION +IF_0 +IF_1 +IN_ATOMIC +INCLUDE_LINUX +INDENTED_LABEL +INLINE +JIFFIES_COMPARISON +KREALLOC_ARG_REUSE +LEADING_SPACE +LIKELY_MISUSE +LINE_CONTINUATIONS +LINE_SPACING +LINUX_VERSION_CODE +LONG_COMMIT_TEXT +LONG_SUMMARY_LINE +LONG_UDELAY +MACRO_WITH_FLOW_CONTROL +MASK_THEN_SHIFT +MDELAY +MEMORY_BARRIER +MEMSET +MINMAX +MISORDERED_TYPE +MISPLACED_INIT +MISSING_BREAK +MISSING_EOF_NEWLINE +MISSING_VMLINUX_SYMBOL +MODULE_LICENSE +MSLEEP +MULTIPLE_DECLARATION +NAKED_SSCANF +NETWORKING_BLOCK_COMMENT_STYLE +NEW_TYPEDEFS +NO_COMMIT_TEXT +NONBLANK_AFTER_SUMMARY +NR_CPUS +ONE_SEMICOLON +OOM_MESSAGE +PATCH_PREFIX +PREFER_ALIGNED +PREFER_DEV_LEVEL +PREFER_ETH_BROADCAST_ADDR +PREFER_ETHER_ADDR_COPY +PREFER_ETHER_ADDR_EQUAL +PREFER_ETH_ZERO_ADDR +PREFER_PACKED +PREFER_PRINTF +PREFER_PR_LEVEL +PREFER_SCANF +PREFER_SEQ_PUTS +PRINTF_L +PRINTK_RATELIMITED +PRINTK_WITHOUT_KERN_LEVEL +QUOTED_WHITESPACE_BEFORE_NEWLINE +RETURN_VOID +SINGLE_STATEMENT_DO_WHILE_MACRO +SIZEOF_ADDRESS +SIZEOF_PARENTHESIS +SPACE_BEFORE_TAB +SPACING +SPLIT_STRING +SSCANF_TO_KSTRTO +STATIC_CONST_CHAR_ARRAY +STORAGE_CLASS +STRING_FRAGMENTS +SUSPECT_CODE_INDENT +TRAILING_SEMICOLON +UNDOCUMENTED_DT_STRING +UNNECESSARY_BREAK +UNNECESSARY_CASTS +UNNECESSARY_ELSE +UNNECESSARY_KERN_LEVEL +UNNECESSARY_PARENTHESES +USE_DEVICE_INITCALL +USE_FUNC +USE_NEGATIVE_ERRNO +USE_RELATIVE_PATH +USE_SPINLOCK_T +USLEEP_RANGE +UTF8_BEFORE_PATCH +VOLATILE +WAITQUEUE_ACTIVE +WHITESPACE_AFTER_LINE_CONTINUATION +YIELD + diff --git a/static_analysis/checkpatch_presubmit.sh b/static_analysis/checkpatch_presubmit.sh new file mode 100755 index 0000000..d854e16 --- /dev/null +++ b/static_analysis/checkpatch_presubmit.sh @@ -0,0 +1,90 @@ +#!/bin/bash + +# Copyright (C) 2019 The Android Open Source Project +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Wrapper around checkpatch.sh to gather necessary information from the +# dist dir. Notably, this includes the git_sha1 and whether to suppress +# the check for post-submit. + +set -e + +export STATIC_ANALYSIS_SRC_DIR=$(dirname $(readlink -f $0)) + +source ${STATIC_ANALYSIS_SRC_DIR}/../envsetup.sh +export OUT_DIR=$(readlink -m ${OUT_DIR:-${ROOT_DIR}/out/${BRANCH}}) +export DIST_DIR=$(readlink -m ${DIST_DIR:-${OUT_DIR}/dist}) + +APPLIED_PROP_PATH=${DIST_DIR}/applied.prop +BUILD_INFO_PATH=${DIST_DIR}/BUILD_INFO + +verify_file_exists() { + if [[ ! -f $1 ]]; then + echo "Missing $1" + exit 1 + fi +} + +# Parse flags. +BUILD_ID="" +FORWARDED_ARGS=() +while [[ $# -gt 0 ]]; do + next="$1" + case ${next} in + --bid) + BUILD_ID="$2" + shift + ;; + --help) + echo "Checks whether given build is for presubmit. If so, extract git_sha1" + echo "from repo.prop and invoke checkpatch.sh." + echo "" + echo "Usage: $0" + echo " <--bid nnn> (The build ID. Required.)" + echo " <args for checkpatch.sh>" + exit 0 + ;; + *) + FORWARDED_ARGS+=("$1") + ;; + esac + shift +done + +if [[ -z $BUILD_ID ]]; then + echo "WARNING: No --bid supplied. Assuming not presubmit build. Exiting." + exit 0 +fi + +# Skip checkpatch for postsubmit (b/35390488). +set +e +echo "${BUILD_ID}" | grep -E "^P[0-9]+" +if [[ $? -ne 0 ]]; then + echo "Did not identify a presubmit build. Exiting." + exit 0 +fi +set -e + +# Pick the correct patch to test. +verify_file_exists ${APPLIED_PROP_PATH} +GIT_SHA1=$(grep -E "${KERNEL_DIR} [0-9a-f]+" "${APPLIED_PROP_PATH}" | awk '{print $2}') +if [[ -z ${GIT_SHA1} ]]; then + # Since applied.prop only tracks user changes, ignore projects that are + # included in presubmit without any changed files. + echo "No changes to apply for ${KERNEL_DIR}." + exit 0 +fi + +${STATIC_ANALYSIS_SRC_DIR}/checkpatch.sh --git_sha1 ${GIT_SHA1} ${FORWARDED_ARGS[*]} + |