Using an Artificial Decision Maker (ADM) to Compare Interactive Evolutionary Methods¶

This tutorial demonstrates how to use Artificial Decision Makers (ADMs) to compare interactive multiobjective evolutionary optimization methods. We will focus on comparing two methods:

1. Interactive RVEA (Reference Vector Guided Evolutionary Algorithm)
2. Interactive NSGA-III (Non-dominated Sorting Genetic Algorithm III)

We will use the ADM proposed by Afsar et al. and evaluate performance using the R-metric framework:

• R-IGD (R-metric Inverted Generational Distance)
• R-HV (R-metric Hypervolume)

How the ADM Works¶

The ADM operates in two sequential phases:

1. Learning Phase:

   • Explores the entire objective space
   • Generates reference points in least explored areas
   • Builds understanding of the Pareto front
   • Identifies potential regions of interest

2. Decision Phase:

   • Focuses on the identified ROI
   • Refines solutions through targeted reference points
   • Guides the search towards most preferred solutions

Iteration Process¶

At each iteration, the ADM follows these steps:

1. Combines solutions from all algorithms and removes dominated ones
2. Divides the combined front using uniform reference vectors
3. Maps solutions to reference vectors based on angular distance
4. Generates new reference points based on current phase
5. Evaluates algorithm performance within the ROI

Reference:
Afsar, B., Miettinen, K., & Ruiz, A. B. (2021).
An Artificial Decision Maker for Comparing Reference Point Based Interactive Evolutionary Multiobjective Optimization Methods. In: Ishibuchi, H., et al. Evolutionary Multi-Criterion Optimization. EMO 2021. Lecture Notes in Computer Science, vol 12654. Springer, Cham.

Problem Definition¶

For this demonstration, we will use the DTLZ2 benchmark with 3 objective functions and 12 decision variables.

First, we need to import the required modules and create the problem instance:

In [1]:

import numpy as np

from desdeo.adm.ADMAfsar import ADMAfsar
from desdeo.emo.hooks.archivers import NonDominatedArchive
from desdeo.emo.methods.EAs import nsga3, rvea
from desdeo.problem.testproblems import dtlz2
from desdeo.tools.indicators_unary import r_metric_indicator

problem = dtlz2(n_objectives=3, n_variables=12)

import numpy as np from desdeo.adm.ADMAfsar import ADMAfsar from desdeo.emo.hooks.archivers import NonDominatedArchive from desdeo.emo.methods.EAs import nsga3, rvea from desdeo.problem.testproblems import dtlz2 from desdeo.tools.indicators_unary import r_metric_indicator problem = dtlz2(n_objectives=3, n_variables=12)

Initializing ADM and Optimization Methods¶

The procedure consists of the following steps:

1. Initialize ADM with the selected problem and specify the number of iterations for each phase.
   • Learning phase: 4 iterations
   • Decision phase: 3 iterations
2. Generate an initial reference point randomly within the objective space.
3. Run both interactive NSGA-III and interactive RVEA using the same reference point.

Note: The initial iteration with the random reference point is not included in the phase iterations.

In [2]:

adm = ADMAfsar(problem=problem, it_learning_phase=4, it_decision_phase=3, number_of_vectors=100)

symbols = [f"{x.symbol}" for x in problem.objectives]

solver_nsga3, publisher_nsga3 = nsga3(
    problem=problem,
    reference_vector_options={
        "reference_point": dict(zip(symbols, adm.preference, strict=False)),
        "interactive_adaptation": "reference_point",
        "number_of_vectors": 100,
    },
)
solver_rvea, publisher_rvea = rvea(
    problem=problem,
    reference_vector_options={
        "reference_point": dict(zip(symbols, adm.preference, strict=False)),
        "interactive_adaptation": "reference_point",
        "number_of_vectors": 100,
    },
)

archive_nsga3 = NonDominatedArchive(problem=problem, publisher=publisher_nsga3)
archive_rvea = NonDominatedArchive(problem=problem, publisher=publisher_rvea)
publisher_nsga3.auto_subscribe(archive_nsga3)
publisher_rvea.auto_subscribe(archive_rvea)

results_nsga3 = solver_nsga3()
results_rvea = solver_rvea()

adm = ADMAfsar(problem=problem, it_learning_phase=4, it_decision_phase=3, number_of_vectors=100) symbols = [f"{x.symbol}" for x in problem.objectives] solver_nsga3, publisher_nsga3 = nsga3( problem=problem, reference_vector_options={ "reference_point": dict(zip(symbols, adm.preference, strict=False)), "interactive_adaptation": "reference_point", "number_of_vectors": 100, }, ) solver_rvea, publisher_rvea = rvea( problem=problem, reference_vector_options={ "reference_point": dict(zip(symbols, adm.preference, strict=False)), "interactive_adaptation": "reference_point", "number_of_vectors": 100, }, ) archive_nsga3 = NonDominatedArchive(problem=problem, publisher=publisher_nsga3) archive_rvea = NonDominatedArchive(problem=problem, publisher=publisher_rvea) publisher_nsga3.auto_subscribe(archive_nsga3) publisher_rvea.auto_subscribe(archive_rvea) results_nsga3 = solver_nsga3() results_rvea = solver_rvea()

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Interactive Optimization Process¶

The ADM framework steers the optimization through an automated, interactive procedure:

Phase Control¶

• adm.has_next(): Checks whether additional iterations are required
• adm.get_next_reference_point(): Generates the next reference point based on the current set of solutions

Phase-Specific Behavior¶

1. Learning Phase

   • Explores diverse regions of the objective space
   • Identifies promising areas for subsequent exploration

2. Decision Phase

   • Concentrates on the region of interest identified during the learning phase
   • Produces targeted reference points to refine the search

Performance Evaluation¶

At each iteration, solution quality is evaluated using R-IGD and R-HV.

The following cell demonstrates this iterative process in practice.

In [3]:

import pandas as pd

iteration = 0
metrics_data = []
reference_points = []

while adm.has_next():
    iteration += 1
    front_rvea = results_rvea.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy()
    front_nsga3 = results_nsga3.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy()

    adm.get_next_preference(front_rvea, front_nsga3)
    reference_points.append(adm.preference.copy())
    solver_nsga3, publisher_nsga3 = nsga3(
        problem=problem,
        reference_vector_options={
            "reference_point": dict(zip(symbols, adm.preference, strict=False)),
            "interactive_adaptation": "reference_point",
            "number_of_vectors": 100,
        },
    )
    solver_rvea, publisher_rvea = rvea(
        problem=problem,
        reference_vector_options={
            "reference_point": dict(zip(symbols, adm.preference, strict=False)),
            "interactive_adaptation": "reference_point",
            "number_of_vectors": 100,
        },
    )
    results_nsga3 = solver_nsga3()
    results_rvea = solver_rvea()

    # Calculate metrics
    metrics_nsga3 = r_metric_indicator(
        results_nsga3.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy(), np.array([adm.preference])
    )
    metrics_rvea = r_metric_indicator(
        results_rvea.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy(), np.array([adm.preference])
    )

    # Store metrics in the list
    current_metrics = pd.DataFrame(
        [
            {
                "Iteration": iteration,
                "Algorithm": "NSGA-III",
                "R-IGD": metrics_nsga3.r_igd,  # First value is R-IGD
                "R-HV": metrics_nsga3.r_hv,  # Second value is R-HV
                "Phase": "Learning" if iteration <= 4 else "Decision",  # noqa: PLR2004
            },
            {
                "Iteration": iteration,
                "Algorithm": "RVEA",
                "R-IGD": metrics_rvea.r_igd,  # First value is R-IGD
                "R-HV": metrics_rvea.r_hv,  # Second value is R-HV
                "Phase": "Learning" if iteration <= 4 else "Decision",  # noqa: PLR2004
            },
        ]
    )

    if len(metrics_data) == 0:
        metrics_data = current_metrics
    else:
        metrics_data = pd.concat([metrics_data, current_metrics], ignore_index=True)

    # Print current metrics in a formatted table
    print(f"\nIteration {iteration} ({current_metrics['Phase'].iloc[0]} Phase)")  # noqa: T201
    print("-" * 70)  # noqa: T201
    print(current_metrics.to_string(index=False, float_format=lambda x: f"{x:.6f}"))  # noqa: T201
    print("-" * 70)  # noqa: T201

    # Print reference point
    ref_point = np.array(adm.preference)
    print(f"Reference Point: [{', '.join(f'{x:.6f}' for x in ref_point)}]")  # noqa: T201

import pandas as pd iteration = 0 metrics_data = [] reference_points = [] while adm.has_next(): iteration += 1 front_rvea = results_rvea.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy() front_nsga3 = results_nsga3.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy() adm.get_next_preference(front_rvea, front_nsga3) reference_points.append(adm.preference.copy()) solver_nsga3, publisher_nsga3 = nsga3( problem=problem, reference_vector_options={ "reference_point": dict(zip(symbols, adm.preference, strict=False)), "interactive_adaptation": "reference_point", "number_of_vectors": 100, }, ) solver_rvea, publisher_rvea = rvea( problem=problem, reference_vector_options={ "reference_point": dict(zip(symbols, adm.preference, strict=False)), "interactive_adaptation": "reference_point", "number_of_vectors": 100, }, ) results_nsga3 = solver_nsga3() results_rvea = solver_rvea() # Calculate metrics metrics_nsga3 = r_metric_indicator( results_nsga3.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy(), np.array([adm.preference]) ) metrics_rvea = r_metric_indicator( results_rvea.optimal_outputs.select(["f_1", "f_2", "f_3"]).to_numpy(), np.array([adm.preference]) ) # Store metrics in the list current_metrics = pd.DataFrame( [ { "Iteration": iteration, "Algorithm": "NSGA-III", "R-IGD": metrics_nsga3.r_igd, # First value is R-IGD "R-HV": metrics_nsga3.r_hv, # Second value is R-HV "Phase": "Learning" if iteration <= 4 else "Decision", # noqa: PLR2004 }, { "Iteration": iteration, "Algorithm": "RVEA", "R-IGD": metrics_rvea.r_igd, # First value is R-IGD "R-HV": metrics_rvea.r_hv, # Second value is R-HV "Phase": "Learning" if iteration <= 4 else "Decision", # noqa: PLR2004 }, ] ) if len(metrics_data) == 0: metrics_data = current_metrics else: metrics_data = pd.concat([metrics_data, current_metrics], ignore_index=True) # Print current metrics in a formatted table print(f"\nIteration {iteration} ({current_metrics['Phase'].iloc[0]} Phase)") # noqa: T201 print("-" * 70) # noqa: T201 print(current_metrics.to_string(index=False, float_format=lambda x: f"{x:.6f}")) # noqa: T201 print("-" * 70) # noqa: T201 # Print reference point ref_point = np.array(adm.preference) print(f"Reference Point: [{', '.join(f'{x:.6f}' for x in ref_point)}]") # noqa: T201

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Iteration 1 (Learning Phase)
----------------------------------------------------------------------
 Iteration Algorithm    R-IGD     R-HV    Phase
         1  NSGA-III 0.001120 8.037481 Learning
         1      RVEA 0.010375 7.935104 Learning
----------------------------------------------------------------------
Reference Point: [0.000000, 1.001452, 0.000000]

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Iteration 2 (Learning Phase)
----------------------------------------------------------------------
 Iteration Algorithm    R-IGD     R-HV    Phase
         2  NSGA-III 0.011180 8.712228 Learning
         2      RVEA 0.003860 8.760101 Learning
----------------------------------------------------------------------
Reference Point: [0.877004, 0.501145, 0.125286]

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Iteration 3 (Learning Phase)
----------------------------------------------------------------------
 Iteration Algorithm    R-IGD     R-HV    Phase
         3  NSGA-III 0.006779 8.688912 Learning
         3      RVEA 0.007577 8.671025 Learning
----------------------------------------------------------------------
Reference Point: [0.585911, 0.000000, 0.820276]

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Iteration 4 (Learning Phase)
----------------------------------------------------------------------
 Iteration Algorithm    R-IGD     R-HV    Phase
         4  NSGA-III 0.006539 8.550080 Learning
         4      RVEA 0.019311 8.319422 Learning
----------------------------------------------------------------------
Reference Point: [0.000000, 0.900709, 0.450354]

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Iteration 5 (Decision Phase)
----------------------------------------------------------------------
 Iteration Algorithm    R-IGD     R-HV    Phase
         5  NSGA-III 0.004815 8.701948 Decision
         5      RVEA 0.007930 8.583002 Decision
----------------------------------------------------------------------
Reference Point: [0.123151, 0.492605, 0.862059]

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Iteration 6 (Decision Phase)
----------------------------------------------------------------------
 Iteration Algorithm    R-IGD     R-HV    Phase
         6  NSGA-III 0.004815 8.701948 Decision
         6      RVEA 0.007930 8.583002 Decision
----------------------------------------------------------------------
Reference Point: [0.123151, 0.492605, 0.862059]

/home/docs/checkouts/readthedocs.org/user_builds/desdeo/checkouts/latest/desdeo/emo/methods/EAs.py:78: UserWarning: Adaptation frequency was set to 0. Setting it to 100 for RVEA selector. Set it to 0 only if you provide preference information.
  selector = RVEASelector(

Iteration 7 (Decision Phase)
----------------------------------------------------------------------
 Iteration Algorithm    R-IGD     R-HV    Phase
         7  NSGA-III 0.004815 8.701948 Decision
         7      RVEA 0.007930 8.583002 Decision
----------------------------------------------------------------------
Reference Point: [0.123151, 0.492605, 0.862059]

Visualization of Reference Points¶

Let's visualize how the ADM generates reference points throughout both phases of the optimization process. The following plot shows:

1. Learning Phase (Left): The first 4 reference points (L1-L4) demonstrate how the ADM explores different regions of the Pareto front to understand the full range of available trade-offs.

2. Decision Phase (Right): The subsequent 3 reference points (D1-D3) show how the ADM focuses on a specific region of interest, refining the search based on the knowledge gained during the learning phase.

The gray surface represents the Pareto front of the DTLZ2 problem, which forms a quarter-sphere in the first octant (all objectives ≥ 0). Points are labeled chronologically to show the sequence of reference point generation.

In [4]:

import numpy as np
import plotly.graph_objects as go
import plotly.io as pio
from plotly.subplots import make_subplots

pio.renderers.default = "notebook"

reference_points = np.array(reference_points)
phi = np.linspace(0, np.pi / 2, 50)
theta = np.linspace(0, np.pi / 2, 50)
phi, theta = np.meshgrid(phi, theta)

x = np.cos(theta) * np.cos(phi)
y = np.cos(theta) * np.sin(phi)
z = np.sin(theta)

# Separate reference points by phase
learning_points = reference_points[:4]  # First 4 iterations
decision_points = reference_points[4:]  # Remaining iterations

# Create figure with two subplots
fig = make_subplots(
    rows=1, cols=2, specs=[[{"type": "scene"}, {"type": "scene"}]], subplot_titles=("Learning phase", "Decision phase")
)

# Add Pareto front to both views
fig.add_trace(
    go.Surface(x=x, y=y, z=z, colorscale="Greys", showscale=False, opacity=0.3, name="Pareto Front"), row=1, col=1
)
fig.add_trace(
    go.Surface(x=x, y=y, z=z, colorscale="Greys", showscale=False, opacity=0.3, name="Pareto Front"), row=1, col=2
)

# Add learning points
fig.add_trace(
    go.Scatter3d(
        x=learning_points[:, 0],
        y=learning_points[:, 1],
        z=learning_points[:, 2],
        mode="markers+text",
        marker={"size": 8, "color": "blue"},
        text=[f"L{i + 1}" for i in range(len(learning_points))],
        name="Learning Phase",
        showlegend=True,
    ),
    row=1,
    col=1,
)

# Add decision points
fig.add_trace(
    go.Scatter3d(
        x=decision_points[:, 0],
        y=decision_points[:, 1],
        z=decision_points[:, 2],
        mode="markers+text",
        marker={"size": 8, "color": "red"},
        text=[f"D{i + 1}" for i in range(len(decision_points))],
        name="Decision Phase",
        showlegend=True,
    ),
    row=1,
    col=2,
)

# Update layout
fig.update_layout(
    title="Reference Points Generared by the ADM",
    width=1200,
    height=600,
    showlegend=True,
)

fig.show()

import numpy as np import plotly.graph_objects as go import plotly.io as pio from plotly.subplots import make_subplots pio.renderers.default = "notebook" reference_points = np.array(reference_points) phi = np.linspace(0, np.pi / 2, 50) theta = np.linspace(0, np.pi / 2, 50) phi, theta = np.meshgrid(phi, theta) x = np.cos(theta) * np.cos(phi) y = np.cos(theta) * np.sin(phi) z = np.sin(theta) # Separate reference points by phase learning_points = reference_points[:4] # First 4 iterations decision_points = reference_points[4:] # Remaining iterations # Create figure with two subplots fig = make_subplots( rows=1, cols=2, specs=[[{"type": "scene"}, {"type": "scene"}]], subplot_titles=("Learning phase", "Decision phase") ) # Add Pareto front to both views fig.add_trace( go.Surface(x=x, y=y, z=z, colorscale="Greys", showscale=False, opacity=0.3, name="Pareto Front"), row=1, col=1 ) fig.add_trace( go.Surface(x=x, y=y, z=z, colorscale="Greys", showscale=False, opacity=0.3, name="Pareto Front"), row=1, col=2 ) # Add learning points fig.add_trace( go.Scatter3d( x=learning_points[:, 0], y=learning_points[:, 1], z=learning_points[:, 2], mode="markers+text", marker={"size": 8, "color": "blue"}, text=[f"L{i + 1}" for i in range(len(learning_points))], name="Learning Phase", showlegend=True, ), row=1, col=1, ) # Add decision points fig.add_trace( go.Scatter3d( x=decision_points[:, 0], y=decision_points[:, 1], z=decision_points[:, 2], mode="markers+text", marker={"size": 8, "color": "red"}, text=[f"D{i + 1}" for i in range(len(decision_points))], name="Decision Phase", showlegend=True, ), row=1, col=2, ) # Update layout fig.update_layout( title="Reference Points Generared by the ADM", width=1200, height=600, showlegend=True, ) fig.show()