Model Info

This repository contains a machine learning model developed by NTT DATA as part of the MLSysOps project.

This artifact provides an ONNX export (opset 18) of a Deep Q-Network (DQN) agent trained to select the best data center (among 3) for a client request in a 5G/MEC setting, optimizing latency-related outcomes (and incorporating carbon intensity as a feature).
Given the current state of three candidate data centers, the model outputs Q-values for each possible selection and chooses the data center with the highest Q-value.

Purpose

This model performs a Reinforcement Learning agent that predict a discrete value:

• Input: A normalized tensor representing the current state of three candidate data centers, shaped as (3 × 10), where each row corresponds to a data center and each column represents a feature such as client identifier, resource utilization, network metrics, latency statistics, packet loss, and carbon intensity. The input is MinMax-scaled using parameters learned during training and includes a label-encoded client identifier.
• Output: A discrete action in {0, 1, 2} corresponding to the selection of the optimal data center among the three candidates, computed as the index with the highest predicted Q-value.

Repository Structure

A typical layout (as used in the accompanying repository/bundle):

• model/

  • 5g_latency_opt_dqn_model.onnx — ONNX model (DQN Q-network)

  • model_config.json — model metadata, I/O specs, feature order, preprocessing parameters

• src/

  • state_serializer.py — builds the model input tensor from JSON scenarios and applies preprocessing

  • minmax_scaler.py — lightweight MinMax scaling implementation (training-fitted parameters)

  • inference_engine.py — ONNXRuntime inference wrapper (argmax over Q-values)

  • action_interpreter.py — converts predicted action to a human-readable decision

• demo.py — end-to-end demo (JSON → preprocess → ONNX inference → decoded action)

• requirements.txt — minimal Python dependencies

Training Data

The model was trained on a tabular dataset containing per-data-center telemetry and network metrics. Each decision step groups 3 rows (one per candidate data center) into a single observation.

Features Used

The training preprocessing:

• MinMax normalization for numeric features:

  • cpu_usage_percent

  • memory_usage_percent

  • disk_usage_percent

  • net_in_percent

  • net_out_percent

  • latency_avg

  • latency_mdev

  • lost_percent

  • carbon_intensity

• Label encoding for:

  • client_id

• Dropped columns:

  • start_time, end_time, net_in_absolute, net_out_absolute,

  • latency_min, latency_max

Important: the inference pipeline must reuse the same MinMaxScaler parameters (data_min/data_max) and the same client_id encoding mapping fitted during training.

Model Architecture

The exported ONNX model contains the SB3 DQN policy Q-network (MLP-based Q-function). The network maps a (3×10) observation (three candidate data centers, ten features each) to Q-values for the three discrete actions (select DC0/DC1/DC2).
At inference time, the recommended decision is argmax(Q-values).

Model Specification

Inputs

• Name: observation

• Type: float32

• Shape: (batch_size, 3, 10)
  where:

  • dimension 1 = candidate data centers (always 3)

  • dimension 2 = feature vector per data center

Feature Order (Last Dimension)

The last dimension (size 10) follows this exact order:

1. client_id (label-encoded)

2. cpu_usage_percent (MinMax-scaled)

3. memory_usage_percent (MinMax-scaled)

4. disk_usage_percent (MinMax-scaled)

5. net_in_percent (MinMax-scaled)

6. net_out_percent (MinMax-scaled)

7. latency_avg (MinMax-scaled)

8. latency_mdev (MinMax-scaled)

9. lost_percent (MinMax-scaled)

10. carbon_intensity (MinMax-scaled)

Outputs

• Name: q_values

• Type: float32

• Shape: (batch_size, 3)

• Meaning: Q-values for the three actions:

  • action 0 → select Data Center 0 (Milan)

  • action 1 → select Data Center 1 (Rome)

  • action 2 → select Data Center 2 (Cosenza)

Limitations

• The model is trained for exactly 3 candidate data centers; input shape is fixed to (3,10).

• Correct behavior depends on identical preprocessing:

  • MinMax scaling must use training-fitted min/max values

  • client_id must be encoded using the training-fitted mapping (unknown IDs should be handled explicitly)

• Generalization outside the training distribution (different telemetry ranges, unseen client populations, different operational conditions) is not guaranteed.

• This model provides a decision policy but does not guarantee optimality; it should be validated in the target deployment setting before use.

Usage Demo

Setup Environment

Create a Python environment and install dependencies:

At minimum, the runtime requires:

• onnxruntime

• numpy

• pandas

Run Inference Script

Run the end-to-end demo:

The demo will:

1. Load a JSON scenario containing dataCenterStates (3 entries)

2. Apply preprocessing (MinMax scaling + client_id encoding)

3. Run ONNX inference via ONNXRuntime

4. Print the chosen data center index (argmax over Q-values)

Citation

If you use this model/artifact in academic work, please cite it as: