Sobre

Goal Design and implement a streamlined CI/CD pipeline for the LadybugDB open‑source repo that efficiently builds and publishes Python wheels and CLI binaries for multiple platforms (Linux/macOS, x86_64/arm64). Reduce redundant compilation across arches/pythons, leverage caching or prebuilt artifacts, and formalize release tagging/publishing. Current pain points - Wheels and binaries are built separately per architecture/python, causing duplicated full builds. - Cache usage is ad‑hoc; no clear artifact reuse strategy. - Release flow (tag → build → publish) is brittle and slow. Deliverables 1. CI/CD design and implementation (GitHub Actions preferred) that: - Builds Python wheels for CPython 3.12, 3.13, 3.14, plus CLI binaries for Linux and macOS (x86_64 & arm64). - Minimizes duplicated compilation (e.g., single “prebuilt core”/SDK artifact reused by wheel and binary jobs, or cross-job caching). - Uses reliable caching (ccache, build/stage artifacts) with clear keys and retention. - Supports tag-driven releases (tag push triggers build & publish). 2. Publishing steps: - Upload wheels to PyPI (or Google Artifact Registry—configurable). - Attach binaries to GitHub Releases. 3. Local developer workflow: - Document how to run CI jobs locally with act or equivalent smoke tests. - Minimal README/CONTRIBUTING section for releasing (version bump, tag, trigger). 4. Optional but desired: - Matrix that can be reduced for PRs (e.g., smoke build) and expanded for tags (full release). - Sane timeouts/retries and failure diagnostics. Tech context - Codebase: LadybugDB (fork of Kuzu), CMake/Ninja build, pybind11 wheels via cibuildwheel. - Current private fork already created a more simpler CI/CD focused on our needs. - Runners: GitHub Actions; act available for local dry-runs. What to submit - Code (privately sent) with updated workflows, supporting scripts, and docs. - A short note on build times before/after and how duplication was eliminated. - Instructions to publish a new version (commands + required secrets/vars). Must-have experience - cibuildwheel, multi-arch wheel builds, CMake/Ninja. - GitHub Actions artifact caching strategies (ccache, tarball SDKs). - Packaging native Python extensions and release automation. Budget/timeline - I need this in at most 72 hours