From 9df4bc9013c2075f16f668e77c23e7e6393c04b5 Mon Sep 17 00:00:00 2001 From: Michael Katsoulis Date: Mon, 12 Jan 2026 10:56:52 +0100 Subject: [PATCH] update untracked files --- .gitignore | 4 +- README.md | 138 ++++++++++++++++++++--------------------- exwayz_test.dockerfile | 114 +++++++++++++++++----------------- notes.md | 32 +++++----- pkgs/README.md | 22 +++---- 5 files changed, 155 insertions(+), 155 deletions(-) diff --git a/.gitignore b/.gitignore index 0be0a0b..1c12169 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,3 @@ -licenses/** -pkgs/exwayz_3d_mapping* +licenses/** +pkgs/exwayz_3d_mapping* tmp/** \ No newline at end of file diff --git a/README.md b/README.md index d0c8a11..4da415e 100644 --- a/README.md +++ b/README.md @@ -1,69 +1,69 @@ -# **Project Overview** - -This repository contains the container and supporting files used by the LENA autonomous vehicle project. It packages the Exwayz 3D Mapping / SLAM executables into a Docker image that can run as part of the vehicle microservices stack. - -**Repository Contents** - -- **`exwayz_test.dockerfile`**: Dockerfile used to build the container image for testing and integration. -- **`src/**`: all code that we have written which runs on the container. -- **`tmp/data/`**: sample data including `exwayz_sample_data_slam_01.bag` used for demonstrations and tests. Download this from the link to their google drive on [the Exwayz docs site](https://exwayz.notion.site/Exwayz-3D-Mapping-v1-latest-1ea57c917367810ea1e9f8bf83e2e9d4) -- **`media/`**: screenshots and demos (GIFs) used in documentation. -- **`licenses/`**: licensing information for included components. - -**Contributing** - -Please open issues or PRs for fixes and improvements. If you work on vehicle deployment or CI, include small, testable changes and document any required secrets or hardware assumptions. - - -# **Quick Start** - -Build the image (from the repository root): - -```powershell -docker build -f exwayz_test.dockerfile -t exwayz_test:latest . -``` - -Run the container (example; adjust mounts and environment for your platform): - -```powershell -# Example (WSL on systems with X11 forwarding available) -docker run -it --rm \ - --env="DISPLAY=$DISPLAY" \ - --env="QT_X11_NO_MITSHM=1" \ - --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \ - -v "${PWD}:/app" \ - exwayz_test:latest \ - /bin/bash -``` - -On POSIX shells use `$DISPLAY` and `$(pwd)` as usual. - -Run the Exwayz SLAM pipeline against the sample bag inside the container (example): - -```bash -exwayz_slam --bag tmp/data/exwayz_sample_data_slam_01.bag \ - --bag-hesai /hesai/pandar_packets \ - --hesai-model PandarXT-32 \ - --d-write 0.1 -o ./slam_session_tutorial --no-date -``` - -Adjust paths and model names to match your sensor and bag file layout. - -You should see something like this: -![Gif of demo](media\Exwayz_slam_demo_working.gif) - -# **Status** - -- Docker image: mostly working for local testing. -- Secrets: currently not included in the image; build-time and runtime secrets need secure handling (see TODOs). -- Target: this repo should evolve into the deploy-and-run repo for vehicle microservices. - -# **Todos** -- [x] **Prototype Docker Image**: Get a prototype docker image working and run their demos. -- [x] **Refine README**: polish documentation and examples (in progress). -- [ ] **Fix Cross Compiling**: Cross compiling failing for aarch64. -- [ ] **Secrets Handling**: implement secure secret injection for builds and runtime (Docker BuildKit, secret manager, or CI secrets). -- [ ] **Standardized Scripts**: add `build_image.ps1` and `run_container.ps1` (and POSIX equivalents) to standardize workflows. -- [ ] **CI Build**: add GitHub Actions workflow equivalent to Gitea to build (and optionally scan) images on push/PR. -- [ ] **Vehicle Deployment Guide**: document runtime flags, service templates, and health-checks for deployment on the vehicle. -- [ ] **Integration Test**: add tests that run the pipeline against `tmp/data/exwayz_sample_data_slam_01.bag` to detect regressions. +# **Project Overview** + +This repository contains the container and supporting files used by the LENA autonomous vehicle project. It packages the Exwayz 3D Mapping / SLAM executables into a Docker image that can run as part of the vehicle microservices stack. + +**Repository Contents** + +- **`exwayz_test.dockerfile`**: Dockerfile used to build the container image for testing and integration. +- **`src/**`: all code that we have written which runs on the container. +- **`tmp/data/`**: sample data including `exwayz_sample_data_slam_01.bag` used for demonstrations and tests. Download this from the link to their google drive on [the Exwayz docs site](https://exwayz.notion.site/Exwayz-3D-Mapping-v1-latest-1ea57c917367810ea1e9f8bf83e2e9d4) +- **`media/`**: screenshots and demos (GIFs) used in documentation. +- **`licenses/`**: licensing information for included components. + +**Contributing** + +Please open issues or PRs for fixes and improvements. If you work on vehicle deployment or CI, include small, testable changes and document any required secrets or hardware assumptions. + + +# **Quick Start** + +Build the image (from the repository root): + +```powershell +docker build -f exwayz_test.dockerfile -t exwayz_test:latest . +``` + +Run the container (example; adjust mounts and environment for your platform): + +```powershell +# Example (WSL on systems with X11 forwarding available) +docker run -it --rm \ + --env="DISPLAY=$DISPLAY" \ + --env="QT_X11_NO_MITSHM=1" \ + --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \ + -v "${PWD}:/app" \ + exwayz_test:latest \ + /bin/bash +``` + +On POSIX shells use `$DISPLAY` and `$(pwd)` as usual. + +Run the Exwayz SLAM pipeline against the sample bag inside the container (example): + +```bash +exwayz_slam --bag tmp/data/exwayz_sample_data_slam_01.bag \ + --bag-hesai /hesai/pandar_packets \ + --hesai-model PandarXT-32 \ + --d-write 0.1 -o ./slam_session_tutorial --no-date +``` + +Adjust paths and model names to match your sensor and bag file layout. + +You should see something like this: +![Gif of demo](media\Exwayz_slam_demo_working.gif) + +# **Status** + +- Docker image: mostly working for local testing. +- Secrets: currently not included in the image; build-time and runtime secrets need secure handling (see TODOs). +- Target: this repo should evolve into the deploy-and-run repo for vehicle microservices. + +# **Todos** +- [x] **Prototype Docker Image**: Get a prototype docker image working and run their demos. +- [x] **Refine README**: polish documentation and examples (in progress). +- [ ] **Fix Cross Compiling**: Cross compiling failing for aarch64. +- [ ] **Secrets Handling**: implement secure secret injection for builds and runtime (Docker BuildKit, secret manager, or CI secrets). +- [ ] **Standardized Scripts**: add `build_image.ps1` and `run_container.ps1` (and POSIX equivalents) to standardize workflows. +- [ ] **CI Build**: add GitHub Actions workflow equivalent to Gitea to build (and optionally scan) images on push/PR. +- [ ] **Vehicle Deployment Guide**: document runtime flags, service templates, and health-checks for deployment on the vehicle. +- [ ] **Integration Test**: add tests that run the pipeline against `tmp/data/exwayz_sample_data_slam_01.bag` to detect regressions. diff --git a/exwayz_test.dockerfile b/exwayz_test.dockerfile index a3d679e..00eff48 100644 --- a/exwayz_test.dockerfile +++ b/exwayz_test.dockerfile @@ -1,57 +1,57 @@ -# syntax=docker/dockerfile:1.3 -FROM ros:jazzy-ros-core - -# Build argument provided by buildx/BuildKit: `TARGETARCH` (amd64, arm64, etc.). -# Default to amd64 when not provided. -ARG TARGETARCH=amd64 -ARG GRAPICAL_ENV=TRUE - -# Copy both architecture-specific packages into the image and choose the right one at build time -COPY pkgs/exwayz_3d_mapping-1.2.2-linux-u24.04-amd64.deb /tmp/exwayz_amd64.deb -COPY pkgs/exwayz_3d_mapping-1.2.2-linux-u24.04-aarch64.deb /tmp/exwayz_aarch64.deb - -# Install dependencies and setup for GUI applications -ENV DISPLAY=:0 -ENV WAYLAND_DISPLAY=wayland-0 -ENV XDG_SESSION_TYPE=x11 -ENV XDG_RUNTIME_DIR=/mnt/wslg/runtime-dir -#If debug, install the following: -RUN if [ "$GRAPHICAL_ENV" = "TRUE" ]; then \ - apt update && apt install -y libgl1-mesa-dev libosmesa6-dev libglew-dev libgles2-mesa-dev libglfw3-dev libstdc++6 mesa-utils; \ - fi - #TODO we're installing unneccessary things like the humanity-icons package, clean this up later. - - -# RUN --mount=type=secret,id=exwayz_license set -eux; \ -# if [ "${TARGETARCH}" = "aarch64" ]; then \ -# pkg=/tmp/exwayz_aarch64.deb; \ -# else \ -# pkg=/tmp/exwayz_amd64.deb; \ -# fi; \ -# apt-get update; \ -# apt-get install -fy "$pkg"; \ -# if [ -s /run/secrets/exwayz_license ]; then \ -# exwayz_license --query "$(cat /run/secrets/exwayz_license)" || true; \ -# fi; \ -# rm -f /tmp/exwayz_amd64.deb /tmp/exwayz_aarch64.deb; \ -# rm -rf /var/lib/apt/lists/* - -COPY licenses/exwayz_liscence.lic /tmp/exwayz_license -RUN set -eux; \ - apt-get update; \ - apt-get install -fy /tmp/exwayz_amd64.deb; \ - if [ -f /tmp/exwayz_license ]; then \ - exwayz_license --query "$(cat /tmp/exwayz_license)" || true; \ - else \ - echo "No license file provided"; \ - exit 1; \ - fi; \ - rm -f /tmp/exwayz_amd64.deb /tmp/exwayz_aarch64.deb; \ - rm -f /tmp/exwayz_license; \ - rm -rf /var/lib/apt/lists/* - -# # Change the working directory to /app -# WORKDIR /app - -# Set entrypoint -ENTRYPOINT ["/ros_entrypoint.sh"] +# syntax=docker/dockerfile:1.3 +FROM ros:jazzy-ros-core + +# Build argument provided by buildx/BuildKit: `TARGETARCH` (amd64, arm64, etc.). +# Default to amd64 when not provided. +ARG TARGETARCH=amd64 +ARG GRAPICAL_ENV=TRUE + +# Copy both architecture-specific packages into the image and choose the right one at build time +COPY pkgs/exwayz_3d_mapping-1.2.2-linux-u24.04-amd64.deb /tmp/exwayz_amd64.deb +COPY pkgs/exwayz_3d_mapping-1.2.2-linux-u24.04-aarch64.deb /tmp/exwayz_aarch64.deb + +# Install dependencies and setup for GUI applications +ENV DISPLAY=:0 +ENV WAYLAND_DISPLAY=wayland-0 +ENV XDG_SESSION_TYPE=x11 +ENV XDG_RUNTIME_DIR=/mnt/wslg/runtime-dir +#If debug, install the following: +RUN if [ "$GRAPHICAL_ENV" = "TRUE" ]; then \ + apt update && apt install -y libgl1-mesa-dev libosmesa6-dev libglew-dev libgles2-mesa-dev libglfw3-dev libstdc++6 mesa-utils; \ + fi + #TODO we're installing unneccessary things like the humanity-icons package, clean this up later. + + +# RUN --mount=type=secret,id=exwayz_license set -eux; \ +# if [ "${TARGETARCH}" = "aarch64" ]; then \ +# pkg=/tmp/exwayz_aarch64.deb; \ +# else \ +# pkg=/tmp/exwayz_amd64.deb; \ +# fi; \ +# apt-get update; \ +# apt-get install -fy "$pkg"; \ +# if [ -s /run/secrets/exwayz_license ]; then \ +# exwayz_license --query "$(cat /run/secrets/exwayz_license)" || true; \ +# fi; \ +# rm -f /tmp/exwayz_amd64.deb /tmp/exwayz_aarch64.deb; \ +# rm -rf /var/lib/apt/lists/* + +COPY licenses/exwayz_liscence.lic /tmp/exwayz_license +RUN set -eux; \ + apt-get update; \ + apt-get install -fy /tmp/exwayz_amd64.deb; \ + if [ -f /tmp/exwayz_license ]; then \ + exwayz_license --query "$(cat /tmp/exwayz_license)" || true; \ + else \ + echo "No license file provided"; \ + exit 1; \ + fi; \ + rm -f /tmp/exwayz_amd64.deb /tmp/exwayz_aarch64.deb; \ + rm -f /tmp/exwayz_license; \ + rm -rf /var/lib/apt/lists/* + +# # Change the working directory to /app +# WORKDIR /app + +# Set entrypoint +ENTRYPOINT ["/ros_entrypoint.sh"] diff --git a/notes.md b/notes.md index 2fc7083..3a9fce3 100644 --- a/notes.md +++ b/notes.md @@ -1,17 +1,17 @@ -## Secrets -Secrets currently not working. -I ran the liscencing script inside the container manually and all worked as expected. -Need to debug and see how we can do this on build. - -## Data -Pulled data from the Exwayz demo on notion's [drive link](https://drive.google.com/drive/folders/1oC_iTZT5e3NsNjVr1Tsyn-uNB1uE5P68). -Placed it here: `tmp\data\exwayz_sample_data_slam_01.bag` - -When running demo with this repo mounted at `/app` it seems to have worked well. -Had a full successful run. See [gif](media\Exwayz_slam_demo_working.gif). -![Gif of demo](media\Exwayz_slam_demo_working.gif) - - -## Cross arch compatibility -Started working on cross arch compat. Builds for `aarch64` taking much longer as expected. +## Secrets +Secrets currently not working. +I ran the liscencing script inside the container manually and all worked as expected. +Need to debug and see how we can do this on build. + +## Data +Pulled data from the Exwayz demo on notion's [drive link](https://drive.google.com/drive/folders/1oC_iTZT5e3NsNjVr1Tsyn-uNB1uE5P68). +Placed it here: `tmp\data\exwayz_sample_data_slam_01.bag` + +When running demo with this repo mounted at `/app` it seems to have worked well. +Had a full successful run. See [gif](media\Exwayz_slam_demo_working.gif). +![Gif of demo](media\Exwayz_slam_demo_working.gif) + + +## Cross arch compatibility +Started working on cross arch compat. Builds for `aarch64` taking much longer as expected. But failing... not as expected. \ No newline at end of file diff --git a/pkgs/README.md b/pkgs/README.md index 81d456e..cad5721 100644 --- a/pkgs/README.md +++ b/pkgs/README.md @@ -1,12 +1,12 @@ -This is where you want to put the packages from Exwayz. - -Download them from the latest relaease on [Exwayz 3D Mapping releases](https://exwayz.notion.site/Exwayz-3D-Mapping-releases-19057c917367807bb19de8007c4243cc) - -Alternatively, if these links still work you need: -### Linux (Ubuntu 24.04) - -**x86_64:** [exwayz_3d_mapping-1.2.2-linux-u24.04-amd64.deb](https://exwayz-releases.s3.eu-west-3.amazonaws.com/1.2.2/exwayz_3d_mapping-1.2.2-linux-u24.04-amd64.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYS2NSS62CY7QV437%2F20260103%2Feu-west-3%2Fs3%2Faws4_request&X-Amz-Date=20260103T021624Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=f6fba24ace0a163eafbebed4a0a0d19949f8e5c384222d32d440de0fd2df4209 -) - -**ARMv8:** [exwayz_3d_mapping-1.2.2-linux-u24.04-aarch64.deb](https://exwayz-releases.s3.eu-west-3.amazonaws.com/1.2.2/exwayz_3d_mapping-1.2.2-linux-u24.04-aarch64.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYS2NSS62CY7QV437%2F20260103%2Feu-west-3%2Fs3%2Faws4_request&X-Amz-Date=20260103T021622Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=d051d59e1339001affde1527ceace4d4f851348f4a5ff40b96e86bbb1b4c29bc +This is where you want to put the packages from Exwayz. + +Download them from the latest relaease on [Exwayz 3D Mapping releases](https://exwayz.notion.site/Exwayz-3D-Mapping-releases-19057c917367807bb19de8007c4243cc) + +Alternatively, if these links still work you need: +### Linux (Ubuntu 24.04) + +**x86_64:** [exwayz_3d_mapping-1.2.2-linux-u24.04-amd64.deb](https://exwayz-releases.s3.eu-west-3.amazonaws.com/1.2.2/exwayz_3d_mapping-1.2.2-linux-u24.04-amd64.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYS2NSS62CY7QV437%2F20260103%2Feu-west-3%2Fs3%2Faws4_request&X-Amz-Date=20260103T021624Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=f6fba24ace0a163eafbebed4a0a0d19949f8e5c384222d32d440de0fd2df4209 +) + +**ARMv8:** [exwayz_3d_mapping-1.2.2-linux-u24.04-aarch64.deb](https://exwayz-releases.s3.eu-west-3.amazonaws.com/1.2.2/exwayz_3d_mapping-1.2.2-linux-u24.04-aarch64.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYS2NSS62CY7QV437%2F20260103%2Feu-west-3%2Fs3%2Faws4_request&X-Amz-Date=20260103T021622Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=d051d59e1339001affde1527ceace4d4f851348f4a5ff40b96e86bbb1b4c29bc ) \ No newline at end of file