Cherry-Pick 2.14.2 SDK changes (#12123)

* update kfp server api for kfp sdk (#12120)

with the recent proto change the kfp server no long accepts body as an
acceptable parameter, due to this breaking change the min kfp-server-api
in kfp sdk needs to be bumped and a new kfp sdk needs to be released.

Signed-off-by: Humair Khan <[email protected]>

* chore: update release instructions (#12117)

Signed-off-by: Humair Khan <[email protected]>

---------

Signed-off-by: Humair Khan <[email protected]>

Author: HumairAK

RELEASE.md

@@ -4,12 +4,12 @@
   - [Schedule](#schedule)
   - [Release Tags and Branches](#release-tags-and-branches)
   - [Contributor Instructions](#contributor-instructions)
-    - [Cherry picking pull requests to release branch](#cherry-picking-pull-requests-to-release-branch)
+    - [Cherry-picking pull requests to release branch](#cherry-picking-pull-requests-to-release-branch)
       - [Option - (Recommended) cherrypick-approved label](#option---recommended-cherrypick-approved-label)
       - [Option - git cherry-pick](#option---git-cherry-pick)
   - [Release Manager Instructions](#release-manager-instructions)
     - [Common Prerequisites](#common-prerequisites)
-    - [Cutting a release branch (Optional)](#cutting-a-release-branch-optional)
+    - [Cutting a release branch (Optional)](#cutting-a-release-branch)
     - [Before release](#before-release)
     - [Releasing from release branch](#releasing-from-release-branch)
     - [Release KFP Python Packages](#releasing-kfp-python-packages)
@@ -97,7 +97,10 @@ if you only want to use or contribute to this repo.
     1. Clone github.com/kubeflow/pipelines repo into `$KFP_REPO`.
     2. `cd $KFP_REPO`
 
-### Cutting a release branch (Optional)
+### Cutting a release branch
+
+KFP releases are required to be cut from a release branch. This includes all python packages.
+Release branches must be scoped to a minor version. The following components should always have all future patch versions tagged and released from their respective minor branch. For example, you would cut a release for SDK 2.14.1 from the release-2.14 branch, the same with KFP backend 2.14.4, kfp-kubernetes 2.14.2, and so on.
 
 1. Choose a good commit on master branch with commit hash as `$COMMIT_SHA`.
 1. Choose the next release branch's `$MINOR_VERSION` in format `x.y`, e.g. `1.0`, `1.1`...
@@ -113,9 +116,9 @@ if you only want to use or contribute to this repo.
 ### Before release
 
 Do the following things before a release:
-1. **(Do this step only when releasing from a NON-master release branch)**
+1. Cherry-picking
 
-    Note: Instead of following this step to cherry pick all PRs, you can also manually cherry pick commits from master branch to release branch, if the number of PRs to cherry pick is minimal. Command for manual cherry pick:
+    Note: Instead of following this step to cherry-pick all PRs, you can also manually cherry-pick commits from the master branch to release branch, if the number of PRs to cherry-pick is minimal. Command for manual cherry-pick:
 
     ```
     git cherry-pick <commit-id>
@@ -177,7 +180,6 @@ If not, contact the KFP team to determine if the failure(s) would block the rele
 
 ### Releasing from release branch
 
-Note, when releasing from master, all the below mentions of "release branch" means master branch.
 
 1. Choose the release's complete `$VERSION` following semantic versioning, e.g.
     - `1.0.0-rc.1`
@@ -193,8 +195,8 @@ Note, when releasing from master, all the below mentions of "release branch" mea
     ```bash
     cd ./test/release && TAG=$VERSION BRANCH=$BRANCH make release
     ```
-
-    It will prompt you whether to push it to release branch. Press `y` and hit `Enter`.
+    This script updates the version values for various manifests, and generated code.
+    Once finished, it will prompt you whether to push it to release branch. You can inspect the changes by navigating to the temporary directory it creates. Once you are comfortable with the changes, press `y` and hit `Enter`. 
 
     Note, the script will clone kubeflow/pipelines repo into a temporary location on your computer, make those changes and attempt to push to upstream, so that it won't interfere with your current git repo.
     
@@ -209,51 +211,106 @@ The target tag should be `$VERSION`.
 
 ### Releasing KFP Python Packages
 
-1. Release `kfp-server-api` python packages to PyPI.
+All Python packages must be released with wheel and source packages. When doing a minor release, you *must* make a release for all Python packages as well, even if there are no new changes there. This includes: 
+
+* kfp-pipeline-spec
+* kfp
+* kfp-kubernetes
+* kfp-server-api
+
 
+> [!Note]
+> When making a release, if something goes wrong, always yank the release in pypi, **do not delete** the package and try to re-upload it with the same version, pypi won't let you do this even though it lets you delete the package. In such an event, yank the release and do a new release with a new patch version.
+
+When performing these releases, you should adhere to the order presented below.
+
+1. Release `kfp-pipeline-spec` Python packages to PyPI.
+    Update the version in `setup.py` found in `api/v2alpha1/python/setup.py`. 
     ```bash
-    git checkout $BRANCH
-    git pull upstream $BRANCH
-    cd backend/api/v2beta1/python_http_client
-    rm -r dist
-    python3 setup.py --quiet sdist
-    python3 -m twine upload dist/*
+    git checkout -b release-X.Y
+    pip3 install twine --user
+    cd api
+    make python 
+    cd v2alpha1/python
+    twine check dist/*
+    twine upload dist/*
     ```
 
-1. Release `kfp` python packages to PyPI. (Note: Please skip this step for backend release, this step will be handled by SDK release.)
+1. Release `kfp` Python packages to PyPI.
 
     Update the SDK version in `version.py` and `readthedocs` `versions.json`, example PR [here](https://github.com/kubeflow/pipelines/pull/11715/files).
 
     ```bash
+    git checkout -b release-X.Y
     pip3 install twine --user
-    cd sdk/python
-    ./build.sh kfp-$VERSION.tar.gz
-    python3 -m twine upload kfp-$VERSION.tar.gz
+    cd sdk
+    make python
+    cd python
+    twine check dist/*
+    twine upload dist/*
     ```
 
     !!! The file name must contain the version. See <https://github.com/kubeflow/pipelines/issues/1292>
 
-1. Release `kfp-kubernetes` python packages to PyPI. (Note: Please skip this step for backend release, this step will be handled by SDK release.)
+** Update Readthedocs **
+
+* Create a GitHub release for KFP SDK release. [Here's an example](https://github.com/kubeflow/pipelines/releases/tag/sdk-2.14.1) reference for a template.
+  * When creating a release create a new tag `sdk-x.y.z`
+* Navigate to the readthedocs website [here](https://app.readthedocs.org/projects/kubeflow-pipelines/), login if needed
+* You should see a new build under "Versions" section for this new tag, ensure it succeeds.
+* Click "Settings"
+* Set the default version to `sdk-x.y.z` (the version we just built and released)
+* Set the default branch to be the release branch `release-x.y.z`
+
+1. Release `kfp-kubernetes` Python packages to PyPI.
 
     Update the KFP Kubernetes SDK version in `__init__.py` and `readthedocs` `versions.json`, example PR [here](https://github.com/kubeflow/pipelines/pull/11380).
 
     ```bash
-    export KFP_KUBERNETES_VERSION=
+    git checkout -b release-X.Y
     pip3 install twine --user
+    cd kubernetes_platform
+    make python 
+    cd python
+    twine check dist/*
+    twine upload dist/*
+    
+    # Cut release-the-docs branch 
+    export KFP_KUBERNETES_VERSION= # Set this to the version being released x.y.z
     cd kubernetes_platform/python
     ./create_release_branch.sh
     ```
+    
+    Follow the output push instructions to **commit and push the read the docs release branch to KFP**.
 
-    Follow the output push instructions to **commit and push the branch to KFP**, then do the following:
+> [!Note]
+> Note that kfp-kubernetes package has a separate readthedocs site and requires that a new branch be pushed for readthedocs to be able to host multiple pages from the same repo. 
+> Every new patch version for this package requires us to create a new release branch purely for readthedocs purposes. However always cut this branch from the `release-X.Y` branch.
+
+** Update Readthedocs ** 
+Once the branch is updated, you need to add this version to readthedocs. Follow these steps: 
+
+* Navigate to the package section on the readthedocs website [here](https://app.readthedocs.org/projects/kfp-kubernetes/). 
+* Click "Add version"
+* Enter the branch `kfp-kubernetes-x.y.z` where x.y.z is the version you released, if you pushed it and it's not showing up, press the "Resync Versions" button and try again 
+* Add this version, navigate back to the "Versions" section, and you should see a build, make sure it succeeds before moving onto the next section.
+* Go to Settings
+* Set this version as the default version. 
+* Click Save
+* Click "View Docs" to navigate to the docs page and ensure the new version shows up as the default. 
+   
+1. Release `kfp-server-api` Python packages to PyPI.
 
     ```bash
-    # set this to the appropriate version that matches what was set in __init__.py earlier
-    export KFP_KUBERNETES_VERSION=
-    cd kubernetes_platform/python
-    ./release.sh
+    git checkout $BRANCH
+    git pull upstream $BRANCH
+    cd backend/api/v2beta1/python_http_client
+    rm -r dist
+    python3 setup.py --quiet sdist
+    python3 -m twine upload dist/*
     ```
-   
-    Note that this script will build the package, test install, and push to PyPi.
+
+Push the changes to the `release-X.Y` branch. 
 
 ### Create GitHub Release
 
@@ -269,7 +326,7 @@ fill in the description. Detailed steps:
        <pre>
        To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in [here](https://www.kubeflow.org/docs/components/pipelines/operator-guides/installation/)
 
-       Install python SDK (python 3.9 above) by running:
+       Install Python SDK (Python 3.9 above) by running:
 
        ```bash
        python3 -m pip install kfp kfp-server-api --upgrade
@@ -284,7 +341,7 @@ fill in the description. Detailed steps:
        <pre>
         To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in [here](https://www.kubeflow.org/docs/components/pipelines/operator-guides/installation/).
 
-        Install kfp-server-api package (python 3.9 above) by running:
+        Install kfp-server-api package (Python 3.9 above) by running:
 
         ```bash
         python3 -m pip install kfp-server-api==$VERSION --upgrade
@@ -294,13 +351,12 @@ fill in the description. Detailed steps:
         * [Upgrade Notes with notices and breaking changes](https://www.kubeflow.org/docs/components/pipelines/installation/upgrade/)
         * [Change Log](https://github.com/kubeflow/pipelines/blob/$VERSION/CHANGELOG.md)
 
-        NOTE, kfp python SDK is **NOT** included and released separately.
+        NOTE, kfp Python SDK is **NOT** included and released separately.
        </pre>
 
 ### Sync Master Branch with latest release
 
-1. **(Do this step only when releasing from a NON-master release branch)**
-Update master branch to the same version and include latest changelog:
+1. Update master branch to the same version and include latest changelog:
 
     ```bash
     git checkout master
@@ -350,7 +406,8 @@ Starting from version **2.14**, all major and minor versions (X.Y) of the Kubefl
   All KFP components sharing the same major and minor version (X.Y) are guaranteed to be API-compatible.
 
 * **Backward Compatibility:**
-  The KFP project will make a *best effort* to maintain backward compatibility within a given **major version** for all Python SDK packages.
+  The KFP project aims to maintain backward compatibility within a given **major version** for all Python SDK packages, though there may be exceptions at times.
+
   Specifically:
 
     * Newer versions of the KFP Python SDK within the same major release (e.g., 2.x) should continue to function with older versions of the KFP backend.

kubernetes_platform/python/release.sh

@@ -37,7 +37,7 @@ else
     TARGET_TAR_FILE=kfp-kubernetes-$KFP_KUBERNETES_VERSION.tar.gz
     pushd "$(dirname "$0")"
     dist_dir=$(mktemp -d)
-    python3 setup.py sdist --format=gztar --dist-dir "$dist_dir"
+    python3 setup.py sdist bdist_wheel --dist-dir "$dist_dir"
     cp "$dist_dir"/*.tar.gz $TARGET_TAR_FILE
     popd
     echo "Created package."

sdk/Makefile

@@ -0,0 +1,32 @@
+# Copyright 2025 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Contact one of chensun, HumairAK if this remote image needs an update.
+PREBUILT_REMOTE_IMAGE=ghcr.io/kubeflow/kfp-api-generator:1.2
+
+.PHONY: python
+python: python
+	docker run --interactive --rm \
+		--user $$(id -u):$$(id -g) \
+		-e HOME=/tmp \
+		-v "$$(pwd)/..":"/go/src/github.com/kubeflow/pipelines":z \
+		$(PREBUILT_REMOTE_IMAGE) \
+		sh -c 'cd /go/src/github.com/kubeflow/pipelines/sdk/python && \
+			python3 setup.py sdist bdist_wheel --dist-dir ./dist'
+
+.PHONY: clean-python
+clean-python:
+	rm -rf python/build
+	rm -rf python/dist
+	rm -rf python/kfp.egg-info

sdk/python/build.sh

@@ -475,7 +475,7 @@ def create_experiment(
                 namespace=namespace,
             )
             experiment = self._experiment_api.experiment_service_create_experiment(
-                body=experiment)
+                experiment=experiment)
 
         link = f'{self._get_url_prefix()}/#/experiments/details/{experiment.experiment_id}'
         if auth.is_ipython():
@@ -736,7 +736,7 @@ def run_pipeline(
             runtime_config=job_config.runtime_config,
             service_account=service_account)
 
-        response = self._run_api.run_service_create_run(body=run_body)
+        response = self._run_api.run_service_create_run(run=run_body)
 
         link = f'{self._get_url_prefix()}/#/runs/details/{response.run_id}'
         if auth.is_ipython():
@@ -907,7 +907,7 @@ def create_recurring_run(
             max_concurrency=max_concurrency,
             service_account=service_account)
         return self._recurring_run_api.recurring_run_service_create_recurring_run(
-            body=job_body)
+            recurring_run=job_body)
 
     def _create_job_config(
         self,

sdk/python/kfp/client/client.py

@@ -1 +1 @@
-__version__ = '2.14.1'
+__version__ = '2.14.2'

sdk/python/kfp/version.py

@@ -19,7 +19,7 @@ kfp-pipeline-spec==2.14.0
 # kfp-server-api package is released.
 # Update the lower version when kfp sdk depends on new apis/fields in
 # kfp-server-api
-kfp-server-api>=2.1.0,<3
+kfp-server-api>=2.14.0,<3
 kubernetes>=8.0.0,<31
 # protobuf version should be identical to the one in kfp-pipeline-spec
 # api/v2alpha1/python/requirements.txt

sdk/python/requirements.in

@@ -49,7 +49,7 @@ idna==3.10
     # via requests
 kfp-pipeline-spec==2.14.0
     # via -r requirements.in
-kfp-server-api==2.5.0
+kfp-server-api==2.14.0
     # via -r requirements.in
 kubernetes==30.1.0
     # via -r requirements.in

sdk/python/requirements.txt

@@ -1,4 +1,4 @@
-kfp==2.14.1
+kfp==2.14.2
 junit-xml
 minio
 black

test/sample-test/requirements.in

@@ -83,11 +83,11 @@ jupyter-core==5.8.1
     #   jupyter-client
     #   nbclient
     #   nbformat
-kfp==2.14.1
+kfp==2.14.2
     # via -r requirements.in
 kfp-pipeline-spec==2.14.0
     # via kfp
-kfp-server-api==2.5.0
+kfp-server-api==2.14.0
     # via kfp
 kubernetes==30.1.0
     # via