feat(dockerfile): install Python 3.11 via Miniconda in CI base image

[wuhuizuo]

Summary

Add Miniconda-based Python 3.11 installation to the CI base Dockerfile. This provides tomllib support needed by ticdc (PR #5357).

Changes

• Install Miniconda to /opt/miniconda
• Install python=3.11 via conda
• Symlink python3.11 to /usr/local/bin/python3.11
• Add /opt/miniconda/bin to PATH

Testing

• CI pipeline will rebuild the Jenkins base image
• Verify Python 3.11 is available: /opt/miniconda/bin/python3.11 --version
• Verify tomllib import works: python3.11 -c "import tomllib"

Risk

Low. Miniconda is installed in a separate directory (/opt/miniconda) and does not interfere with existing python27/python3 system packages. The symlink is to a new python3.11 binary that doesn't conflict with existing python or python3 commands.

Ref: FLA-231

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign wuhuizuo for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

Summary
This PR enhances the CI base Docker image by installing Python 3.11 through Miniconda, enabling the use of the tomllib module required by ticdc. The approach uses a Miniconda installer script based on architecture, installs Python 3.11 via conda, symlinks the python3.11 executable, and updates the PATH environment variable. The implementation is straightforward and logically isolated from existing Python installations, which is good. Overall, the change looks functional but could benefit from some improvements around layer caching, idempotency, and minor hygiene.

---

Critical Issues

• Layer Caching and Image Size

  • File: dockerfiles/ci/base/Dockerfile (lines 18-28)

  • Issue: The current RUN command downloads the Miniconda installer, installs Python 3.11, and cleans up the installer in one layer which is good. However, the conda install step will fetch a large environment and potentially update conda itself, which can lead to a large image size and slow rebuilds.

  • Suggestion: Consider pinning the Miniconda installer version instead of Miniconda3-latest to avoid unexpected changes breaking the build. Also, add conda clean -afy after installation to reduce image size:

    RUN ARCH=$([ "$(arch)" = "x86_64" ] && echo x86_64 || echo aarch64); \
        wget "https://repo.anaconda.com/miniconda/Miniconda3-py311_23.3.1-Linux-${ARCH}.sh" -O /tmp/miniconda.sh && \
        bash /tmp/miniconda.sh -b -p /opt/miniconda && \
        rm /tmp/miniconda.sh && \
        /opt/miniconda/bin/conda install -y python=3.11 && \
        /opt/miniconda/bin/conda clean -afy && \
        ln -s /opt/miniconda/bin/python3.11 /usr/local/bin/python3.11

• Symlink Safety and PATH Precedence

  • File: dockerfiles/ci/base/Dockerfile (line 27-28)

  • Issue: Symlinking python3.11 is good, but the Dockerfile adds /opt/miniconda/bin to the front of PATH via ENV. This means the default python and python3 will point to Miniconda’s versions, possibly interfering with existing workflows. This conflicts with the PR description that says no interference is expected.

  • Suggestion: If you want to avoid overriding default python commands, consider adding only the symlink and avoid prepending Miniconda to PATH. Alternatively, use a dedicated environment or explicit calls to /usr/local/bin/python3.11. For example:

    ENV PATH=/opt/miniconda/bin:$PATH
    # or avoid adding PATH and rely on symlink only

    Confirm this aligns with your risk assessment.

---

Code Improvements

• Architecture Detection Logic

  • File: dockerfiles/ci/base/Dockerfile (line 18)

  • Issue: The arch check only considers x86_64 and assumes anything else is aarch64. This may fail on other architectures or if the arch output differs (e.g., amd64 on some systems).

  • Suggestion: Use a more robust detection (e.g., uname -m), and add a fallback or error for unsupported arch:

    RUN ARCH=$(uname -m); \
        if [ "$ARCH" = "x86_64" ] || [ "$ARCH" = "amd64" ]; then ARCH=x86_64; \
        elif [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then ARCH=aarch64; \
        else echo "Unsupported architecture: $ARCH" && exit 1; fi; \
        # continue with wget...

• Explicit Python Version Pinning

  • File: dockerfiles/ci/base/Dockerfile (line 24)
  • Issue: conda install -y python=3.11 installs the latest patch of 3.11, which could break reproducibility over time.
  • Suggestion: Pin to a specific minor version, e.g., python=3.11.4, to ensure stable builds.

---

Best Practices

• Documentation / Comments

  • File: dockerfiles/ci/base/Dockerfile (line 17-28)
  • Issue: The new RUN block has only a minimal comment.
  • Suggestion: Add more detailed comments explaining why Miniconda is installed, the rationale behind symlinking, and the PATH modification. This will help future maintainers.

• Testing Coverage

  • General
  • Issue: The PR description mentions manual verification commands but no automated tests or CI steps validating the Python 3.11 environment or tomllib import.
  • Suggestion: Add a minimal test in the CI pipeline that runs python3.11 -c "import tomllib" or checks the Python version to catch regressions early.

• Style / Formatting

  • File: dockerfiles/ci/base/Dockerfile

  • Issue: The backslashes in the RUN command could be aligned for readability (e.g., one argument per line).

  • Suggestion: Format as:

    RUN ARCH=... && \
        wget ... && \
        bash ... && \
        rm ... && \
        /opt/miniconda/bin/conda install -y python=3.11 && \
        ln -s ... 

---

Overall, this is a useful and needed enhancement. Addressing caching, reproducibility, and architecture detection will improve robustness and maintainability. Adding minimal tests and clarifying PATH usage will reduce unexpected side effects.