Sim Module

Overview

At a Glance

Purpose:

Simulation orchestration, batch execution, and workflow pipelines

Location:

fusion/sim/

Key Files:

batch_runner.py, input_setup.py, network_simulator.py (legacy)

Depends On:

fusion.core, fusion.io, fusion.modules.rl

Used By:

CLI entry points, experiment scripts

The sim module is the orchestration layer for FUSION simulations. It manages how simulations are run (batch processing, multi-process execution, workflow pipelines) while delegating the what (actual simulation logic) to fusion.core.

Important

This module is NOT the simulation engine. It orchestrates simulation runs. The actual discrete event simulation happens in fusion.core.simulation.

When you work here:

  • Adding new batch execution modes

  • Modifying multi-process coordination

  • Creating new workflow pipelines (training, evaluation)

  • Changing how simulation inputs are prepared

Understanding the Module Landscape

FUSION has several modules with overlapping names that can cause confusion. This section clarifies what each does and when to use it.

Warning

Common Confusion Points:

  • fusion/sim/ vs fusion/core/ - orchestration vs simulation logic

  • fusion/sim/ml_pipeline.py vs fusion/pipelines/ - training workflows vs RSA pipelines

  • fusion/sim/utils/ vs fusion/utils/ - sim-specific vs general utilities

  • fusion/sim/input_setup.py vs fusion/io/ - prep vs I/O operations

Module Comparison Table

Module Responsibilities

Module

Purpose

Contains

Example Use

fusion/sim/

Orchestration

Batch runners, workflow pipelines, input preparation

“Run 10 simulations with different Erlang loads”

fusion/core/

Simulation Engine

Event processing, request handling, statistics

“Process this request arrival event”

fusion/pipelines/

RSA Pipelines

Routing strategies, protection, disjoint paths

“Find k-shortest paths with 1+1 protection”

fusion/io/

Data I/O

Topology loading, data export, file operations

“Load NSFNet topology from file”

fusion/reporting/

Presentation

Result formatting, aggregation, console output

“Display simulation results with confidence intervals”

sim vs core: The Key Difference

+------------------+                    +-------------------+
|   fusion/sim     |  orchestrates -->  |   fusion/core     |
| (BatchRunner)    |                    | (SimulationEngine)|
+------------------+                    +-------------------+
       |                                        |
       | "Run these 5 Erlang loads"             | "Process 10,000 requests"
       | "Use 4 parallel processes"             | "Handle arrivals/departures"
       | "Prepare input files"                  | "Track blocking statistics"

Analogy: sim is the project manager coordinating multiple simulations. core is the worker doing the actual simulation work.

ml_pipeline.py vs pipelines Module

This is a naming collision that causes confusion:

Item

Location

Purpose

ml_pipeline.py

fusion/sim/ml_pipeline.py

Workflow: ML model training orchestration (placeholder)

train_pipeline.py

fusion/sim/train_pipeline.py

Workflow: RL agent training orchestration

pipelines module

fusion/pipelines/

RSA: Routing strategies, protection algorithms

The pipelines module handles RSA (Routing and Spectrum Assignment) algorithms. The *_pipeline.py files in sim/ handle workflow orchestration.

Note

The naming is historical. Future refactoring may rename the workflow files to ml_workflow.py and train_workflow.py to reduce confusion.

input_setup.py vs io Module

Item

Purpose

When Used

input_setup.py

Prepare simulation inputs (calls io functions)

Before simulation starts

fusion/io/

Load/Save data (topology, results)

During I/O operations

input_setup.py is a consumer of fusion/io. It calls create_network(), create_pt(), and create_bw_info() from io to prepare everything needed before a simulation run.

Utils Duplication

There are TWO utils packages with different scopes:

Package

Purpose

Key Functions

fusion/utils/

General utilities (logging, config, OS)

get_logger(), setup_logger()

fusion/sim/utils/

Simulation-specific utilities (network, spectrum)

find_path_length(), find_free_slots()

The sim-specific utils contain 31+ functions for:

  • Network analysis (path length, congestion, fragmentation)

  • Spectrum management (free slots, super-channels)

  • Data processing (matrix operations, scaling)

  • Simulation helpers (erlang values, timestamps)

Tip

When adding new utilities:

  • General-purpose (logging, paths, config) -> fusion/utils/

  • Simulation/network-specific -> fusion/sim/utils/

Multi-Processing Architecture

Warning

Terminology Correction: FUSION uses multi-processing (separate processes), NOT multi-threading. The codebase may have legacy references to “threads” but the actual implementation uses multiprocessing.Pool and multiprocessing.Process.

Important

Multi-Processing Limitations (v6.x)

Multi-processing is NOT fully supported across all configurations:

  • RL training/inference may not work correctly with parallel execution

  • ML pipelines are currently single-process only

  • Some utility functions (e.g., modify_multiple_json_values) are single-process only

  • Protection/failure scenarios have limited parallel support

Planned for future release: Full multi-processing support across all features.

Modern Approach (batch_runner.py)

The recommended approach uses multiprocessing.Pool for task-based parallelism:

BatchRunner.run(parallel=True)
        |
        v
+------------------+
| multiprocessing  |
|     Pool(4)      |
+------------------+
        |
+-------+-------+-------+-------+
|       |       |       |       |
v       v       v       v       v
Task 1  Task 2  Task 3  Task 4  Task 5
E=100   E=150   E=200   E=250   E=300
        |
        v
Results aggregated

Characteristics:

  • Each Erlang load becomes a separate task

  • Pool manages process lifecycle automatically

  • Clean serialization via Pool

  • Progress tracking via Manager().dict()

Legacy Approach (network_simulator.py)

The legacy approach spawns one process per configuration:

NetworkSimulator.run()
        |
+-------+-------+-------+
|       |       |       |
v       v       v       v
Process Process Process Process
Config1 Config2 Config3 Config4
        |
        v
Each runs sequential Erlangs internally

Characteristics:

  • Manual process spawning with multiprocessing.Process

  • Sequential Erlang execution within each process

  • Complex state passing (deepcopy to avoid pickling issues)

  • Manual queue/event management

Note

Which to use? Use BatchRunner for new code. NetworkSimulator exists for backward compatibility with legacy experiment scripts.

Data Flow and Architecture

High-Level Flow

CLI / Experiment Script
         |
         v
+--------------------+
| fusion/sim         |
| (Orchestration)    |
+--------------------+
         |
         +---> input_setup.create_input()
         |          |
         |          v
         |     fusion/io (load topology, create PT)
         |
         +---> BatchRunner / NetworkSimulator
                    |
                    v
           +-------------------+
           | fusion/core       |
           | SimulationEngine  |
           +-------------------+
                    |
                    +---> SDNOrchestrator (new) / LegacyHandler (old)
                    |          |
                    |          v
                    |     fusion/pipelines (routing, spectrum)
                    |
                    v
           +-------------------+
           | Results           |
           +-------------------+
                    |
                    v
           fusion/reporting (format, aggregate, export)

Step-by-Step Execution

  1. Configuration Parsing

    # CLI parses config file
config = load_config("simulation.ini")

  2. Input Preparation (input_setup.py)

    # Creates bandwidth info, topology, physical topology
engine_props = create_input(base_fp, engine_props)

# Internally calls:
# - fusion.io.generate.create_bw_info()
# - fusion.io.structure.create_network()
# - fusion.io.generate.create_pt()

  3. Batch Execution (batch_runner.py)

    runner = BatchRunner(config)
results = runner.run(parallel=True)

# For each Erlang load:
# - Creates SimulationEngine from fusion.core
# - Calls engine.run()
# - Collects results

  4. Simulation (fusion.core)

    engine = SimulationEngine(engine_props)
engine.run()

# Processes 10,000+ request events
# Uses SDNOrchestrator for routing/spectrum

  5. Results

    # Results returned to BatchRunner
# Can be aggregated, exported, reported

Key Data Structures

engine_props (dict):

engine_props = {
    # Network
    "network": "NSFNet",
    "topology": nx.Graph,           # NetworkX graph
    "cores_per_link": 7,

    # Traffic
    "erlang": 300.0,
    "arrival_rate": 0.06,
    "holding_time": 5000.0,

    # Spectrum
    "mod_per_bw": {...},            # Modulation -> bandwidth mapping
    "topology_info": {...},          # Physical topology with cores

    # Execution
    "thread_num": "s1",              # Process identifier
    "progress_dict": {...},          # Shared progress state
}

Batch Results (list[dict]):

results = [
    {
        "erlang": 100.0,
        "elapsed_time": 45.2,
        "stats": {
            "blocking_probability": 0.0023,
            "total_requests": 10000,
        }
    },
    # ... more Erlang results
]

Components

batch_runner.py (Modern)

Purpose:

Modern batch simulation orchestrator with parallel execution

Key Class:

BatchRunner

Key Function:

run_batch_simulation()

from fusion.sim import BatchRunner, run_batch_simulation

# Object-oriented approach
runner = BatchRunner(config)
results = runner.run(parallel=True)

# Convenience function
results = run_batch_simulation(config, parallel=True)

Key Methods:

  • prepare_simulation() - Creates input data and topology

  • run_single_erlang() - Runs one Erlang load

  • run_parallel_batch() - Parallel execution via Pool

  • run_sequential_batch() - Sequential execution

  • run() - Main entry point

network_simulator.py (Legacy)

Purpose:

Legacy multi-process control for backward compatibility

Status:

Deprecated - use batch_runner.py for new code

Key Class:

NetworkSimulator

from fusion.sim.network_simulator import NetworkSimulator

# Legacy usage
simulator = NetworkSimulator()
simulator.run(sims_dict)

run_simulation.py (Compatibility)

Purpose:

Backward-compatible entry points

Key Functions:

run_simulation(), run_simulation_pipeline()

from fusion.sim import run_simulation

# Legacy single-run interface
result = run_simulation(config)

# Internally calls BatchRunner with parallel=False

input_setup.py

Purpose:

Prepare all input data before simulation

Key Functions:

create_input(), save_input()

from fusion.sim.input_setup import create_input

# Prepares:
# - Bandwidth info (modulation assumptions)
# - Network topology
# - Physical topology (cores, fiber properties)
engine_props = create_input(base_fp, engine_props)

Integration with io module:

input_setup.create_input()
        |
        +---> fusion.io.generate.create_bw_info()
        +---> fusion.io.structure.create_network()
        +---> fusion.io.generate.create_pt()

Workflow Pipelines

Warning

Beta Status: The ML and evaluation pipelines are in beta. They contain placeholder implementations that will be expanded in future versions.

train_pipeline.py

Purpose:

Bridge between new config system and RL training workflow

Status:

Functional (bridges to legacy RL)

from fusion.sim import train_rl_agent

# Launches RL training via legacy workflow
train_rl_agent(config)

# Internally:
# 1. Extracts config path
# 2. Creates RL environment
# 3. Calls fusion.modules.rl.workflow_runner.run()

ml_pipeline.py

Purpose:

ML model training orchestration

Status:

Placeholder - not implemented

from fusion.sim.ml_pipeline import train_ml_model

# Currently just logs the config
train_ml_model(config)

Note

This file will be expanded as supervised/unsupervised learning features are developed. Currently it only logs that it was invoked.

evaluate_pipeline.py

Purpose:

Evaluation workflow for models and algorithms

Status:

Beta - contains placeholder implementations

Key Class:

EvaluationPipeline

from fusion.sim import EvaluationPipeline

pipeline = EvaluationPipeline(config)
results = pipeline.run_full_evaluation(eval_config)

Placeholder Functions:

The following functions have placeholder implementations:

  • _run_rl_episode() - Returns random dummy results (needs RL env integration)

  • _generate_comparison_plots() - Stub for visualization

  • _generate_excel_report() - Stub for Excel export

These will be implemented as the evaluation framework matures.

Legacy vs Orchestrator

FUSION has two architectural approaches that coexist:

Aspect

Legacy (v5.x)

Orchestrator (v6.x+)

Location

fusion/core/simulation.py internal

fusion/core/orchestrator.py

Pattern

Monolithic handling

Pipeline-based coordination

RSA Logic

Embedded in engine

Delegated to pipelines

Extensibility

Modify engine directly

Add new pipelines

RL Integration

Legacy adapter

Clean policy interface

SDNOrchestrator (New):

# Thin coordination layer
# Does NOT implement algorithm logic
# Delegates to pipelines

orchestrator = SDNOrchestrator(config, pipelines, policy)
result = orchestrator.handle_arrival(request, network_state)

Rules for SDNOrchestrator:

  1. No algorithm logic (K-shortest-path, first-fit, etc.)

  2. No direct numpy access

  3. No hardcoded slicing/grooming logic

  4. Receives NetworkState per call, never stores it

Beta Features and TODOs

Warning

Features in Beta (v6.x):

  • ML training pipelines (placeholder implementations)

  • Evaluation pipelines (partial implementation)

  • Protection/failure scenarios (limited testing)

  • Multi-process RL training

Important

Known Limitations:

  1. Multi-processing not fully supported:

    • RL training/inference may fail in parallel mode

    • Some configs require parallel=False

    • Planned fix: future release

  2. Hardcoded values in some utilities:

    • 256 spectral slots assumed in some functions

    • 6 cores per link assumed in spectrum utilities

    • Will be parameterized in future versions

  3. Placeholder implementations:

    • ml_pipeline.py - Just logs, no training

    • _run_rl_episode() - Returns random values

    • _generate_comparison_plots() - Stub

Development Guide

Getting Started

  1. Read the Understanding the Module Landscape section

  2. Understand the difference between sim and core

  3. Examine batch_runner.py for the modern execution pattern

  4. Run the tests to see example usage

Common Tasks

Adding a new execution mode

  1. Add method to BatchRunner in batch_runner.py

  2. Update run() to route to new mode based on config

  3. Add tests in tests/test_batch_runner.py

Creating a new workflow pipeline

  1. Create new file {name}_pipeline.py in fusion/sim/

  2. Follow pattern from evaluate_pipeline.py

  3. Add to __init__.py exports

  4. Add CLI integration if needed

Modifying input preparation

  1. Edit input_setup.py

  2. Update calls to fusion/io functions as needed

  3. Ensure backward compatibility with existing configs

Configuration

Batch execution options:

[simulation]
erlang_start = 100
erlang_stop = 500
erlang_step = 50
parallel = true
num_processes = 4

Testing

Test Location:

fusion/sim/tests/

Run Tests:

pytest fusion/sim/tests/ -v

Test files:

  • test_batch_runner.py - Modern batch execution

  • test_network_simulator.py - Legacy execution

  • test_run_simulation.py - Compatibility functions

  • test_train_pipeline.py - RL training bridge

Utils tests:

  • tests/test_network.py - Path/congestion utilities

  • tests/test_spectrum.py - Spectrum utilities

  • tests/test_data_utils.py - Data processing

# Run all sim tests
pytest fusion/sim/tests/ -v

# Run with coverage
pytest --cov=fusion.sim fusion/sim/tests/