Migration followup-2#1180
Draft
EbiRider wants to merge 35 commits into
Draft
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| # RPC from squatting a proxy slot for minutes and starving other RPCs. | ||
| wire_payload = unwrap_secrets_for_send(payload) | ||
| if method == "bdev_lvol_transfer_final_step": | ||
| logger.debug("ebi Wire payload: %s", wire_payload) |
| result = migration_controller.cleanup_migration_target(migration.uuid) | ||
| except ValueError as e: | ||
| raise HTTPException(400, str(e)) | ||
| return result |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| from simplyblock_core.models.base_model import BaseModel | ||
|
|
||
|
|
||
| class LVolMigrationGroup(BaseModel): |
| try: | ||
| last_snap = db.get_snapshot_by_id(last_uuid) | ||
| tgt_snap_composite = f"{tgt_node.lvstore}/{_snap_tgt_short_name(last_snap)}" | ||
| except KeyError: |
| last_snap = db.get_snapshot_by_id(last_uuid) | ||
| if last_snap.lvol and last_snap.lvol.node_id == tgt_node.get_id(): | ||
| tgt_snap_composite = last_snap.snap_bdev | ||
| except KeyError: |
| except (ValueError, KeyError) as e: | ||
| try: | ||
| src_rpc.bdev_nvme_detach_controller(ctrl_name) | ||
| except Exception: |
| m = db.get_migration_by_id(mid) | ||
| if m.is_active(): | ||
| return False | ||
| except KeyError: |
| """ | ||
|
|
||
| import time | ||
| from typing import List, Optional, Tuple |
| import time | ||
| from typing import List, Optional, Tuple | ||
|
|
||
| from simplyblock_core import db_controller as db_mod, utils, constants |
| from typing import List, Optional, Tuple | ||
|
|
||
| from simplyblock_core import db_controller as db_mod, utils, constants | ||
| from simplyblock_core.utils import convert_size |
| from simplyblock_core.controllers import migration_controller, tasks_events | ||
| from simplyblock_core.models.cluster import Cluster | ||
| from simplyblock_core.models.job_schedule import JobSchedule | ||
| from simplyblock_core.models.lvol_migration import LVolMigration |
| 'params': kwargs, | ||
| }) | ||
| if method in ("bdev_lvol_transfer_final_step", "bdev_lvol_batch_final_step"): | ||
| logger.debug("ebi Wire payload: %s", wire_payload) |
…er a failed migration even if the node become reachable later
… anastate to inaccessible b4 final step
…d the task runner
EbiRider
force-pushed
the
migration-followup2
branch
from
July 21, 2026 15:26
e584862 to
3706701
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
this is a big PR that touches upon several topics and add new ones
new feature: Batch lvol migration
this is designed for namespaced volumes, since we can't migrate volumes individually from a shared namespace as of now one of our options is migrating every lvol under the shared subsystem for now this feature has a limit of 16 lvols
how to invoke? we added a new argument to the existing migrate commands --batch passing this is required for batch migration and if a migration is called on a namespaced volume the command will fail asking for --batch argument we set this explicitly for clarity and not mistakenly initiate a group migration also the migrate-continue command now expects a group migration id (more to that later) instead of a single migration Id if the --batch is passed to it
what are the related the changes?
for this we added a new task runner exclusively running the batch some of the logic used on the lvol migration is duplicated here on purpose since it's two distinct features and might need to develop differently this include adding this new container to the docker compose, to the collect logs script etc to establish this new container
on the database side we have a group migration entity that tracks this process and has a one to many relationship with migrations (because the batch migrate will initiate and control several single lvol migrations)
other topics this PR touches?
cleanup logic: the code been enhanced, several bugs fixed and duplicates were removed and put into helpers this is used by both the migration runner itself and a new standalone cleanup command
task runner lvol migration: some parts of the flow were reordered for example switching ana state of the source secondary and tertiary to inaccessible BEFORE the final step, also a revert was added in case the final step failed for any reason,
fixing several bugs for the migration of lvols or clones with a tree of ancestry snapshots
adding new rpc in rpc_client this for the batch migration, and also the migration_final_step has a parameter (operation) with default value of migrate this was changed so the call sites have to explicitly pass this value to add more clarity on the operation being executed, also removed the bdev_lvol_final_migration since it was no longer needed