Skip to content

Conversation

@GioeleB00
Copy link
Collaborator

here’s a clean PR description you can paste:

Summary

Modernizes the public API around arrivals, tightens schemas/validation, and
brings the runtime + tests in line with the new modeling. Focus on making
workload generation explicit, predictable, and easy to sweep.

What changed (high level)

  • Arrivals API migration

    • Introduced ArrivalsGenerator (arrivals in SimulationPayload) as the
      single source of truth for workload (replacing old user/RPM knobs).
    • Unified inter-arrival samplers behind general_interarrivals(...).
    • Added EMPIRICAL path for timestamp-driven arrivals.
  • Runtime refactor

    • New ArrivalsGeneratorRuntime producing RequestStates at stochastic gaps.
    • SimulationRunner build/wire/start pipeline updated to the new arrivals
      model; per-run RNG isolation.
  • Sampling infrastructure

    • Implemented families: EXPONENTIAL/POISSON/DETERMINISTIC/UNIFORM and
      LOG_NORMAL/WEIBULL/PARETO/ERLANG (SCV-tuned).
    • Horizon-safe generators (_iid_to_horizon) to avoid overshooting.
    • Robust empirical builder with sorting, origin anchoring, and min-gap clamp.
  • Schemas & validation

    • Stricter Step, Edge, Server/Client/LB, and SimulationPayload
      validations (coherent kinds/ops, positive means/variance, RAM checks,
      non-overlapping outages, event window within horizon, unique IDs, etc.).
  • Sweep utilities

    • Sweep.sweep_on_lambda(...) to iterate lambda_rps over a numeric grid,
      deep-copying payloads per run and creating a fresh simpy.Environment.
  • Analysis surface

    • Public re-exports in asyncflow.analysis, asyncflow.components,
      asyncflow.settings, and root package cleaned and documented.

Tests

  • Unit tests (complete & green)

    • Samplers: horizon truncation, positivity, variability families, empirical
      path (incl. error cases), seeded determinism.
    • Runtime: arrivals dispatcher/flow, edge delivery vs drop, metrics dict,
      queue-theory MMc instability/sanity.
    • Public surface/import tests for __all__ and classes.
  • Integration tests

    • Minimal topologies (single / two servers with LB).
    • Event injection windows: network spikes and server outages.
    • End-to-end KPIs availability (latency stats & throughput series).
  • Tooling quality gates:

    • ruff and mypy compliant (docstrings, line length, protocols,
      precise call signatures, no kwarg/positional mismatches).

Backwards compatibility

  • Breaking: payload now uses arrivals: ArrivalsGenerator (replaces the old
    “users/RPM” knobs). YAML/consumers must set lambda_rps and model.
  • Topology schemas are stricter; invalid configs will now fail validation
    earlier (by design).

Migration notes

  • Update scenarios to:

    arrivals:
      id: rqs-1
      lambda_rps: 20.0
      model: poisson   # or exponential|deterministic|uniform|log_normal|weibull|pareto|erlang|empirical
      # variability: low|medium|high   # only for SCV-tuned families
      # empirical_data: [ ... ]        # only for model: empirical
  • Replace any user/RPM-based helpers with lambda_rps.

  • When sweeping load, use Sweep.sweep_on_lambda(...).

Dev notes

  • RNG access standardized (np.random.Generator) and injected everywhere.
  • Empirical gaps clamp tiny/negative diffs to avoid tight loops.
  • Tests avoid statistical flakiness; assertions use weak properties + seeds.

Checklist

  • API surface updated & re-exported
  • Migrations applied to runner, runtime actors, and samplers
  • Unit + integration tests added and green
  • ruff / mypy clean
  • Minimal docs in code + helpful error messages

@GioeleB00 GioeleB00 merged commit b22f1e2 into develop Sep 14, 2025
2 checks passed
@GioeleB00 GioeleB00 deleted the refactor/simplifying-api branch September 14, 2025 16:40
GioeleB00 added a commit that referenced this pull request Sep 18, 2025
…q measurement, and metrics plumbing (#29)

* setting up the branch

* minor correction

* minor changes

* improved script for linux

* minor change

* Delete docker_fs/.env.dev

* Delete docker_fs/.env.test

* Features/event generator (#1)

* setting up the generator

* poisson-poisson requests sampling

* Delete docker_fs/.env.dev

* Delete docker_fs/.env.test

* changes

* refactor for event generator

* added gaussian truncated generator and unit tests for helpers

* imprved documentantion and simulation time handling

* simulation start with the vent generator

* small refactor + tests for the generation of events

* minor changes

* Update src/app/schemas/simulation_input.py

Co-authored-by: Copilot <[email protected]>

* Update tests/unit/sampler/test_sampler_helper.py

Co-authored-by: Copilot <[email protected]>

* Update scripts/quality-check.sh

Co-authored-by: Copilot <[email protected]>

* Update tests/integration/db_initialization/test_init_models.py

Co-authored-by: Copilot <[email protected]>

* minor changes after PR review

---------

Co-authored-by: Copilot <[email protected]>

* minor changes

* Features/event generator documentation test improvements (#2)

* moving file to correct folder

* Documentation added

* Update requests_generator.md

* test Added, introduced constants for the sampling window

* Update tests/unit/sampler/test_poisson_posson.py

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

* README update

* Improved constants management

* Clean and refactor

* Features/request handler endpoint input (#3)

* Introduction of the input structure for endpoints

* Documentation + foundation of topology input

* Improve Step validation, added unit test for the input

* improved documentation for the input

* Features/definition full payload simulation (#4)

* Improved input structure and pytest

* Improved pytest structure accordingly to the new schema

* definition of the metrics to be measured and update of the simulation input

* improved documentations added rationale behind metrics

* improved pytest logic and code coherence

* Update src/app/core/helpers.py

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

* Features/rqs generator runtime (#5)

* definition of state and RqsGeneratorRuntime

* defined edge runtime and more central logic for sampler

* minor changes

* minor changes

* minor bug fixed

* pytest adapted to the new structure, added pytest for rqs_state

* Update src/app/core/runtime/rqs_generator.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/runtime/rqs_generator.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/runtime/edge.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/runtime/edge.py

Co-authored-by: Copilot <[email protected]>

* Update tests/unit/runtime/test_requests_generator.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/event_samplers/common_helpers.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/event_samplers/common_helpers.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/event_samplers/common_helpers.py

Co-authored-by: Copilot <[email protected]>

* minor changes

---------

Co-authored-by: Copilot <[email protected]>

* Features/client server runtime (#6)

* defined client runtime + tests

* initiated the server runtime plus correction of tghe structure

* code refactor + test added for runtime + documentation

* Update tests/conftest.py

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

* Features/metric sampler and collection (#7)

* defined architecture for the central collector + documentation

* server modification to collect metrics and updated docs

* Completed metric collector for server plus test

* remove web app dependencies and added metrics for rqs latency

* improved docs and improved metric collection

* changes to make the code compatible with new changes

* improved ci against toml changes

* update lock file

* minor change

* Features/metrics elaboration (#8)

* introduction to the analyzer class + tests

* refactor of the analyzer

* Features/load balancer node (#9)

* Added LB structure in the input for the simulation + tests

* introduced LB runtime + documentation

* Added test for LB and small refactor to define the SimRunner

* Features/simulation runner (#10)

* added the first method to the simulation runner

* added methods to handle nodes

* completed simulation runtime + integration test

* Features/integration tests unit tests (#11)

* new integration test + bug fixing

* added integration and unit test

* first tests working

* minor change

* first working example, upgrade docs, upgrade docstrings

* improvements

* path bug fixed

* new readme and guide to build yaml

* Update README.md

* Added pybuilder and unit tests (#12)

* Refactor/change project name plus docs improvement (#13)

* Changed name from fastsim to asyncflow app folder renamed in asyncflow

* improved docs

* minor changes

* Refactor/pypi preparation (#14)

* Reafctor for the folder schemas + defined import for public api

* Defined public api, improved docs

* minor changes

* minor changes

* Features/lb example and docs tutorial (#15)

* minor docs changes

* improved pytoml

* improved docs + LB examples

* sanity ci check

* Ci for main (#16)

* Readme final, docs improved

* Docs improvement, refactor analyzer, example added

* Added scripts for setup, added pytest, improved readme

* Example LB improvements

* CI for main + system tests

* fixing a bug

* bug fixed

* version bump

* version bump

* Feature/event injection input (#19)

* Structure defined for the pydantic model representing Event to be injected

* Pydantic validation in the payload for events part 1

* Definition of the input for the events injection part 2

* minor fixes ruff and mypy compliant

* full tests and docs added for input event injection

* fixed minor bugs

* Feature/event injection runtime (#20)

* Events injection for edges part 1

* Logic + docs for the event injection to simulate a server down

* Added pydantic validation + unit + int tests for eventinjection

* added exaple for event inj yaml + builder added int tests

* improved server model + system tests for eventinjection

* ruff small fix

* Refactor/fixing readme small fixes (#21)

* new readme, ci to have pycov badge, comment on server runtime

* small changes

* bump version

* Structure input for multiprocessing, improved tests, improved description (#23)

* better test coverage, edges deterministic, step server random (#24)

* new sim value

* Feature/multi processing event loop 3 (#25)

* added metric for server, added logic to collect them + tests

* add kpi to collect in settings default

* added metric plot for server and tests

* Sweep introduction + MM1 heory vs observation + tests

* analysis for sweep results + tests

* mm1 notebook complete + server latency

* Add M/M/c (split) queue-theory analyzer, RANDOM load-balancer policy, and end-to-end comparison reports (#26)

* mmc split + notebook mmc split + random lb algo

* tests for mmc

* New Api RqsGenerator ---> ArrivalsGenerator (#27)

* changing api per ArrivalsGenerator and implementing distrib for ArrivalRuntime

* Reafctoring API Arrival generator, tests integrations now working

* Api Arrival generator migration complete with tests

* Updated plots

* Added edges without network latency plus tests (#28)

* Implemented fcfs with tests

* Implemented fcfs algo and validation of mmc models Erlang C

* small changes

---------

Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants