Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions tests/ci/android/AWSLCAndroidTestRunner/app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'

android {
compileSdkVersion 29
buildToolsVersion "29.0.3"
ndkVersion "21.0.6113669"
compileSdkVersion 30
buildToolsVersion "30.0.3"
ndkVersion "21.4.7075529"

def cmake_args = ''
def specific_abis = ''
Expand Down Expand Up @@ -51,7 +51,7 @@ android {
defaultConfig {
applicationId "software.amazon.aws.crypto.awslcandroidtestrunner"
minSdkVersion 21
targetSdkVersion 29
targetSdkVersion 30
versionCode 1
versionName "1.0"

Expand All @@ -68,7 +68,7 @@ android {
externalNativeBuild {
cmake {
path "src/main/cpp/CMakeLists.txt"
version "3.10.2"
version "3.18.1"
}
}

Expand Down
4 changes: 2 additions & 2 deletions tests/ci/android/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Android CI for AWS-LC
AWS-LC wants to ensure that our tests and build work correctly on Android. Insipred from the CRT Team's [`AWSCRTAndroidTestRunner`](https://github.com/awslabs/aws-c-common/tree/main/AWSCRTAndroidTestRunner), this mini Android test harness is intended to be used within our Android CI with AWS Device Farm. The tests will include `crypto_test`, `urandom_test` and `ssl_test`, referenced from regular test dimensions. This app can be tested with the docker image at `tests/ci/docker_images/linux-x86/ubuntu-20.04_android`.
AWS-LC wants to ensure that our tests and build work correctly on Android. Insipred from the CRT Team's [`AWSCRTAndroidTestRunner`](https://github.com/awslabs/aws-c-common/tree/main/AWSCRTAndroidTestRunner), this mini Android test harness is intended to be used within our Android CI with AWS Device Farm. The tests will include `crypto_test`, `urandom_test` and `ssl_test`, referenced from regular test dimensions. This app can be tested with the docker image at `tests/ci/docker_images/linux-x86/ubuntu-24.04_android`.

Our android CI cross-compiles AWS-LC using AWSLCAndroidTestRunner and the NDK toolchains in Codebuild. The current cross-compilation types are `x86_64`,`armeabi-v7a`, `arm64-v8a` for non-FIPS and `arm64-v8a` for FIPS. After cross-compilation is complete, we kick off real time device testing in AWS Device Farm from Codebuild using `tests/ci/kickoff_devicefarm_job.sh`, `boto3`, and the apk artifacts created.

## Setup
The commands below help build the app within our Android CI's docker image. The docker image is only made to support cross-compiling AWS-LC with Android toolchains, building the `AWSLCAndroidTestRunner` apks, and uploading and kicking off the tests in AWS Device Farm. Running and testing on connected Android devices or emulators via `./gradlew cC` isn't configured within the docker image.
1. Assuming all the commands are being run from this folder: `cd tests/ci/android`
2. `docker build -t ubuntu-20.04:android ../docker_images/linux-x86/ubuntu-20.04_android/`
2. `docker build -t ubuntu-20.04:android ../docker_images/linux-x86/ubuntu-24.04_android/`
3. Run the docker image from root of aws-lc. The container needs access to aws-lc's source code to build.
```
cd ../../../
Expand Down
12 changes: 6 additions & 6 deletions tests/ci/cdk/cdk/codebuild/github_ci_android_omnibus.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,39 +13,39 @@ batch:
privileged-mode: true
compute-type: BUILD_GENERAL1_MEDIUM
# TODO(CryptoAlg-1276): replace |620771051181| and |us-west-2| with corresponding env variables.
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_android_latest
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-24.04_android_latest
- identifier: ubuntu2004_android_nonfips_static_release
buildspec: ./tests/ci/codebuild/android/run_android_static_release.yml
env:
type: LINUX_CONTAINER
privileged-mode: true
compute-type: BUILD_GENERAL1_MEDIUM
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_android_latest
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-24.04_android_latest
- identifier: ubuntu2004_android_nonfips_shared_debug
buildspec: ./tests/ci/codebuild/android/run_android_shared_debug.yml
env:
type: LINUX_CONTAINER
privileged-mode: true
compute-type: BUILD_GENERAL1_MEDIUM
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_android_latest
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-24.04_android_latest
- identifier: ubuntu2004_android_nonfips_shared_release
buildspec: ./tests/ci/codebuild/android/run_android_shared_release.yml
env:
type: LINUX_CONTAINER
privileged-mode: true
compute-type: BUILD_GENERAL1_MEDIUM
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_android_latest
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-24.04_android_latest
- identifier: ubuntu2004_android_fips_shared_release
buildspec: ./tests/ci/codebuild/android/run_android_fips_shared.yml
env:
type: LINUX_CONTAINER
privileged-mode: true
compute-type: BUILD_GENERAL1_MEDIUM
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_android_latest
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-24.04_android_latest
- identifier: ubuntu2004_android_fips_static_release
buildspec: ./tests/ci/codebuild/android/run_android_fips_static.yml
env:
type: LINUX_CONTAINER
privileged-mode: true
compute-type: BUILD_GENERAL1_LARGE
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_android_latest
image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-24.04_android_latest
3 changes: 2 additions & 1 deletion tests/ci/docker_images/linux-x86/build_images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,15 @@ docker build -t ubuntu-20.04:clang-7x ubuntu-20.04_clang-7x
docker build -t ubuntu-20.04:clang-8x ubuntu-20.04_clang-8x
docker build -t ubuntu-20.04:clang-9x ubuntu-20.04_clang-9x
docker build -t ubuntu-20.04:clang-10x ubuntu-20.04_clang-10x
docker build -t ubuntu-20.04:android -f ubuntu-20.04_android/Dockerfile ../
docker build -t ubuntu-20.04:clang-7x-bm-framework ubuntu-20.04_clang-7x-bm-framework
docker build -t ubuntu-22.04:base -f ubuntu-22.04_base/Dockerfile ../dependencies
docker build -t ubuntu-22.04:clang-14x-sde ubuntu-22.04_clang-14x-sde
docker build -t ubuntu-22.04:gcc-10x ubuntu-22.04_gcc-10x
docker build -t ubuntu-22.04:gcc-11x ubuntu-22.04_gcc-11x
docker build -t ubuntu-22.04:gcc-12x ubuntu-22.04_gcc-12x
docker build -t ubuntu-22.04:gcc-12x_integration ubuntu-22.04_gcc-12x_integration
docker build -t ubuntu-24.04:base -f ubuntu-24.04_base/Dockerfile ../dependencies
docker build -t ubuntu-24.04:android -f ubuntu-24.04_android/Dockerfile ../
docker build -t amazonlinux-2:base -f amazonlinux-2_base/Dockerfile ../dependencies
docker build -t amazonlinux-2:gcc-7x amazonlinux-2_gcc-7x
docker build -t amazonlinux-2:gcc-7x-intel-sde amazonlinux-2_gcc-7x-intel-sde
Expand Down
2 changes: 1 addition & 1 deletion tests/ci/docker_images/linux-x86/push_images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ tag_and_push_img 'ubuntu-20.04:clang-7x' "${ECS_REPO}:ubuntu-20.04_clang-7x"
tag_and_push_img 'ubuntu-20.04:clang-8x' "${ECS_REPO}:ubuntu-20.04_clang-8x"
tag_and_push_img 'ubuntu-20.04:clang-9x' "${ECS_REPO}:ubuntu-20.04_clang-9x"
tag_and_push_img 'ubuntu-20.04:clang-10x' "${ECS_REPO}:ubuntu-20.04_clang-10x"
tag_and_push_img 'ubuntu-20.04:android' "${ECS_REPO}:ubuntu-20.04_android"
tag_and_push_img 'ubuntu-24.04:android' "${ECS_REPO}:ubuntu-24.04_android"
tag_and_push_img 'ubuntu-20.04:clang-7x-bm-framework' "${ECS_REPO}:ubuntu-20.04_clang-7x-bm-framework"
tag_and_push_img 'ubuntu-20.04:clang-10x_formal-verification-saw-x86_64' "${ECS_REPO}:ubuntu-20.04_clang-10x_formal-verification-saw-x86_64"
tag_and_push_img 'ubuntu-20.04:clang-10x_formal-verification-saw-x86_64-aes-gcm' "${ECS_REPO}:ubuntu-20.04_clang-10x_formal-verification-saw-x86_64-aes-gcm"
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,33 +1,40 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0 OR ISC

FROM ubuntu:20.04
FROM ubuntu:24.04

SHELL ["/bin/bash", "-c"]

ENV DEBIAN_FRONTEND=noninteractive


ENV ANDROID_SDK_URL commandlinetools-linux-8092744_latest
ENV ANDROID_HOME /opt/sdk
ENV GRADLE_VERSION gradle-5.6.4
ENV ANDROID_SDK_URL=commandlinetools-linux-13114758_latest
ENV ANDROID_HOME=/opt/sdk
ENV ANDROID_SDK_ROOT=/opt/sdk
ENV GRADLE_VERSION=gradle-7.6.4
ENV GOROOT=/usr/local/go
ENV PATH="$GOROOT/bin:$PATH"
ENV PATH="$GOROOT/bin:/opt/sdk/cmdline-tools/latest/bin:$PATH"
# ------------------------------------------------------
# --- Android SDK

RUN set -ex && \
apt-get update -y && \
apt-get -y --no-install-recommends upgrade && \
apt-get -y --no-install-recommends install \
python3.8 \
python3.8-venv \
python3.12 \
python3.12-venv \
python3-pip \
openjdk-8-jdk \
openjdk-17-jdk \
perl \
libunwind-dev \
wget \
unzip && \
# Set Java 17 as default
export JAVA17_ALT=$(update-alternatives --list java | grep java-17 | head -1) && \
update-alternatives --set java $JAVA17_ALT && \
# Set Java 17 for SDK manager compatibility
export JAVA_HOME=$(find /usr/lib/jvm -name "*java-17*" -type d | head -1) && \
export PATH=$JAVA_HOME/bin:$PATH && \
# install android-sdk from url source
mkdir /opt/sdk && \
mkdir /opt/sdk/cmdline-tools && \
Expand All @@ -42,17 +49,20 @@ RUN set -ex && \
yes | ./sdkmanager --licenses && \
# Preinstall AWSLCAndroidTestRunner android dependencies, so they don't need to be
# rebuilt for each new gradle build run.
./sdkmanager "ndk;21.0.6113669" \
"build-tools;29.0.3" \
"cmake;3.10.2.4988404" \
"platforms;android-29" && \
./sdkmanager "ndk;28.2.13676358" \
"build-tools;33.0.3" \
"cmake;3.18.1" \
"platforms;android-30" && \
cd /opt && \
wget -q https://services.gradle.org/distributions/${GRADLE_VERSION}-all.zip && \
rm -rf /opt/cmdline-tools-tmp && \
rm -rf /tmp/*
# Preinstall gradle dependencies, so they don't need to be redownloaded in the CI.
COPY linux-x86/ubuntu-20.04_android /tmp/triggerGradleDownloads/
COPY linux-x86/ubuntu-24.04_android /tmp/triggerGradleDownloads/
RUN cd /tmp/triggerGradleDownloads && \
echo "JAVA_HOME=$JAVA_HOME" && \
java -version && \
echo "PATH=$PATH" && \
./gradlew --no-daemon --refresh-dependencies androidDependencies lint && \
rm -rf /tmp/triggerGradleDownloads

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
plugins {
id 'com.android.application' version '7.3.1'
id 'org.jetbrains.kotlin.android' version '1.8.10'
}

allprojects {
repositories {
google()
mavenCentral()
}
}

android {
namespace 'software.amazon.aws.crypto.awslcandroidtestrunner'
compileSdk 32
buildToolsVersion "32.0.0"
ndkVersion "28.2.13676358"

defaultConfig {
minSdkVersion 21
targetSdkVersion 32

testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
}

dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar'])
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.8.10"
implementation 'androidx.appcompat:appcompat:1.1.0'
implementation 'androidx.core:core-ktx:1.2.0'
androidTestImplementation 'androidx.test.ext:junit:1.1.5'
androidTestImplementation 'androidx.test:runner:1.5.2'
androidTestImplementation 'androidx.test:core:1.5.0'
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# http://www.gradle.org/docs/current/userguide/build_environment.html
# Specifies the JVM arguments used for the daemon process.
# The setting is particularly useful for tweaking memory settings.
org.gradle.jvmargs=-Xmx1536m
org.gradle.jvmargs=-Xmx2048m -Djava.awt.headless=true
# When configured, Gradle will run in incubating parallel mode.
# This option should only be used with decoupled projects. More details, visit
# http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects
Expand All @@ -19,5 +19,4 @@ android.useAndroidX=true
android.enableJetifier=true
# Kotlin code style for this project: "official" or "obsolete":
kotlin.code.style=official
org.gradle.daemon=true
org.gradle.parallel=true
org.gradle.parallel=true
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=file\:/opt/gradle-5.6.4-all.zip
distributionUrl=file\:/opt/gradle-7.6.4-all.zip

# Use version below when building outside of our android docker image and gradle
# is not preinstalled.
# distributionUrl=https\://services.gradle.org/distributions/gradle-5.6.4-all.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.4-all.zip
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
pluginManagement {
repositories {
google()
mavenCentral()
gradlePluginPortal()
}
}
rootProject.name = 'triggerGradleDownloads'
55 changes: 55 additions & 0 deletions tests/ci/docker_images/linux-x86/ubuntu-24.04_base/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0 OR ISC

FROM ubuntu:24.04

SHELL ["/bin/bash", "-c"]

ENV DEBIAN_FRONTEND=noninteractive
ENV DEPENDENCIES_DIR=/home/dependencies
ENV LLVM_PROJECT_HOME=${DEPENDENCIES_DIR}/llvm-project
ENV GOROOT=/usr/local/go
ENV PATH="$GOROOT/bin:$PATH"
ENV ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer

# llvm, llvm-dev, libcxx, and libcxxabi are needed for the sanitizer tests.
# 11.1.0 is the latest stable release as of 2021-02-16.
# See https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo
RUN set -ex && \
apt-get update && \
apt-get -y --no-install-recommends upgrade && \
apt-get -y --no-install-recommends install \
software-properties-common \
cmake \
ninja-build \
perl \
libunwind-dev \
pkg-config \
git \
ca-certificates \
wget \
lld \
llvm \
llvm-dev \
curl \
unzip && \
# Based on https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html
# The awscli is used to publish data to CloudWatch Metrics in some jobs. This requires additional IAM permission
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
unzip awscliv2.zip && \
./aws/install --bin-dir /usr/bin && \
rm -rf awscliv2.zip aws/ && \
# Download a copy of LLVM's libcxx which is required for building and running with Memory Sanitizer
mkdir -p ${DEPENDENCIES_DIR} && \
cd ${DEPENDENCIES_DIR} && \
git clone https://github.com/llvm/llvm-project.git --branch llvmorg-11.1.0 --depth 1 && \
cd llvm-project && rm -rf $(ls -A | grep -Ev "(libcxx|libcxxabi)") && \
apt-get --purge remove -y curl unzip && \
apt-get autoremove --purge -y && \
apt-get clean && \
apt-get autoclean && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /tmp/*

COPY install_common_dependencies.sh /
RUN set -ex && /install_common_dependencies.sh && rm install_common_dependencies.sh
Loading