Evaluating and Comparing VRP Solutions

This guide demonstrates how to use the VRP solver’s evaluation capabilities to assess existing routes and compare them with optimized solutions. This workflow is particularly useful when you want to:

Validate manual route assignments against optimal solutions
Import routes from legacy systems and assess their efficiency
Compare the impact of different business constraints
Understand the cost of manual adjustments to optimized routes

Overview of the Evaluation Workflow

Generate an optimized solution

First, use the standard solve endpoint to get an optimal solution as your baseline.

Create alternative assignments

Make manual adjustments or import existing routes from another system.

Evaluate the alternative

Use the evaluate endpoint to assess the quality of your alternative solution.

Compare results

Analyze scores, constraint violations, and performance metrics between solutions.

Step 1: Generate an Optimized Solution

Start by creating an optimized solution using the standard solve endpoint:

cURL

curl -X POST 'https://api.solvice.io/v2/vrp/solve' \
    -H 'Authorization: YOUR_API_KEY' \
    -H 'Content-Type: application/json' \
    -d '{
  "resources": [
    {
      "name": "Driver-1",
      "shifts": [{
        "from": "2024-01-15T08:00:00Z",
        "to": "2024-01-15T17:00:00Z",
        "start": {"latitude": 50.8503, "longitude": 4.3517}
      }],
      "capacity": [100]
    },
    {
      "name": "Driver-2",
      "shifts": [{
        "from": "2024-01-15T08:00:00Z",
        "to": "2024-01-15T17:00:00Z",
        "start": {"latitude": 50.8466, "longitude": 4.3528}
      }],
      "capacity": [100]
    }
  ],
  "jobs": [
    {
      "name": "Delivery-001",
      "location": {"latitude": 50.8798, "longitude": 4.7005},
      "duration": 600,
      "load": [10],
      "windows": [{
        "from": "2024-01-15T09:00:00Z",
        "to": "2024-01-15T12:00:00Z"
      }]
    },
    {
      "name": "Delivery-002",
      "location": {"latitude": 50.8366, "longitude": 4.3684},
      "duration": 900,
      "load": [15],
      "windows": [{
        "from": "2024-01-15T10:00:00Z",
        "to": "2024-01-15T14:00:00Z"
      }]
    },
    {
      "name": "Delivery-003",
      "location": {"latitude": 50.8939, "longitude": 4.3412},
      "duration": 450,
      "load": [8]
    }
  ],
  "options": {
    "distanceMatrixType": "OSM_ROUTE"
  }
}'

Success Response

{
  "id": "job_12345",
  "status": "PENDING",
  "createdAt": "2024-01-15T07:30:00Z"
}

Save the job ID to retrieve the optimized solution later. The solve operation runs asynchronously.

Step 2: Retrieve the Optimized Solution

Once the job completes, retrieve the solution details:

cURL

curl -X GET 'https://api.solvice.io/v2/vrp/jobs/job_12345/solution' \
    -H 'Authorization: YOUR_API_KEY'

Optimized Solution

{
  "status": "OPTIMAL",
  "score": {
    "hard": 0,
    "medium": 0,
    "soft": -15420
  },
  "statistics": {
    "totalTravelTime": 3842,
    "totalDistance": 45.3
  },
  "jobs": [
    {
      "name": "Delivery-001",
      "resource": "Driver-1",
      "arrival": "2024-01-15T09:15:23Z"
    },
    {
      "name": "Delivery-002",
      "resource": "Driver-1",
      "arrival": "2024-01-15T10:45:12Z"
    },
    {
      "name": "Delivery-003",
      "resource": "Driver-2",
      "arrival": "2024-01-15T08:25:45Z"
    }
  ]
}

Step 3: Create an Alternative Solution

Now, let’s say you want to evaluate a different assignment - perhaps based on customer preferences or manual planning. Create an evaluation request with your alternative assignments:

cURL

curl -X POST 'https://api.solvice.io/v2/vrp/evaluate' \
    -H 'Authorization: YOUR_API_KEY' \
    -H 'Content-Type: application/json' \
    -d '{
  "resources": [
    {
      "name": "Driver-1",
      "shifts": [{
        "from": "2024-01-15T08:00:00Z",
        "to": "2024-01-15T17:00:00Z",
        "start": {"latitude": 50.8503, "longitude": 4.3517}
      }],
      "capacity": [100]
    },
    {
      "name": "Driver-2",
      "shifts": [{
        "from": "2024-01-15T08:00:00Z",
        "to": "2024-01-15T17:00:00Z",
        "start": {"latitude": 50.8466, "longitude": 4.3528}
      }],
      "capacity": [100]
    }
  ],
  "jobs": [
    {
      "name": "Delivery-001",
      "location": {"latitude": 50.8798, "longitude": 4.7005},
      "duration": 600,
      "load": [10],
      "windows": [{
        "from": "2024-01-15T09:00:00Z",
        "to": "2024-01-15T12:00:00Z"
      }],
      "initialResource": "Driver-2",
      "initialArrival": "2024-01-15T09:30:00Z",
      "plannedArrival": "2024-01-15T09:15:23Z"
    },
    {
      "name": "Delivery-002",
      "location": {"latitude": 50.8366, "longitude": 4.3684},
      "duration": 900,
      "load": [15],
      "windows": [{
        "from": "2024-01-15T10:00:00Z",
        "to": "2024-01-15T14:00:00Z"
      }],
      "initialResource": "Driver-2",
      "initialArrival": "2024-01-15T10:45:00Z",
      "plannedArrival": "2024-01-15T10:45:12Z"
    },
    {
      "name": "Delivery-003",
      "location": {"latitude": 50.8939, "longitude": 4.3412},
      "duration": 450,
      "load": [8],
      "initialResource": "Driver-1",
      "initialArrival": "2024-01-15T08:30:00Z",
      "plannedArrival": "2024-01-15T08:25:45Z"
    }
  ],
  "options": {
    "distanceMatrixType": "OSM_ROUTE"
  }
}'

All jobs must have both initialResource and initialArrival fields populated for the evaluate endpoint to work correctly.

Step 4: Compare the Results

The evaluate endpoint returns a similar response structure, allowing direct comparison:

Alternative Solution Evaluation

{
  "status": "FEASIBLE",
  "score": {
    "hard": 0,
    "medium": 0,
    "soft": -18965
  },
  "statistics": {
    "totalTravelTime": 4823,
    "totalDistance": 52.7,
    "deviationFromPlanned": 892
  },
  "jobs": [
    {
      "name": "Delivery-001",
      "resource": "Driver-2",
      "arrival": "2024-01-15T09:30:00Z",
      "plannedDeviation": 867
    },
    {
      "name": "Delivery-002",
      "resource": "Driver-2",
      "arrival": "2024-01-15T10:45:00Z",
      "plannedDeviation": -12
    },
    {
      "name": "Delivery-003",
      "resource": "Driver-1",
      "arrival": "2024-01-15T08:30:00Z",
      "plannedDeviation": 255
    }
  ]
}

Understanding the Comparison

Score Analysis

Optimized Solution
Alternative Solution

Hard Score: 0 (all constraints satisfied)
Soft Score: -15,420 (lower is better)
Travel Time: 3,842 seconds

Key Metrics to Compare

Total Travel Time

The alternative solution requires 981 additional seconds (16 minutes) of travel time.

Distance Traveled

The alternative covers 7.4 km more distance due to less optimal routing.

Planned Deviation

When plannedArrival is provided, the solver tracks how much the actual arrival deviates.

Resource Utilization

Compare how balanced the workload is across drivers in each solution.

Advanced Evaluation Features

Evaluating Partial Solutions

If you only want to fix some assignments and let the solver optimize the rest:

{
  "resources": [
    {
      "name": "Driver-1",
      "shifts": [{
        "from": "2024-01-15T08:00:00Z",
        "to": "2024-01-15T17:00:00Z"
      }]
    }
  ],
  "jobs": [
    {
      "name": "VIP-Delivery",
      "duration": 1800,
      "initialResource": "Driver-1",
      "initialArrival": "2024-01-15T09:00:00Z"
      // This job is fixed to Driver-1
    },
    {
      "name": "Regular-Delivery",
      "duration": 1800
      // This job will be optimized by the solver
    }
  ]
}

Use the suggest endpoint instead of evaluate when you have partially assigned solutions and want recommendations for unassigned jobs.

Getting Detailed Explanations

Enable explanations to understand why certain assignments might be suboptimal:

cURL

curl -X GET 'https://api.solvice.io/v2/vrp/jobs/job_12345/explanation' \
    -H 'Authorization: YOUR_API_KEY'

This returns detailed constraint violation information, helping you understand:

Which constraints are causing score penalties
Why certain assignments might be infeasible
The relative importance of different optimization factors

Best Practices

When to Use Evaluation

Validating manual route plans before execution
Importing and assessing routes from legacy systems
Understanding the impact of business rule changes
Comparing different optimization strategies

Performance Considerations

The evaluate endpoint has similar performance to solve
Use the same distance matrix configuration for fair comparison
Consider using synchronous endpoints for small instances

Common Pitfalls

Forgetting to set initialResource for all jobs
Using inconsistent time formats between requests
Not accounting for distance matrix differences
Comparing solutions with different constraint configurations

Next Steps

Learn about constraint configuration to understand score components
Explore job relations for complex routing scenarios
Check out synchronous operations for real-time evaluation

​Evaluating and Comparing VRP Solutions

​Overview of the Evaluation Workflow

​Step 1: Generate an Optimized Solution

​Step 2: Retrieve the Optimized Solution

​Step 3: Create an Alternative Solution

​Step 4: Compare the Results

​Understanding the Comparison

​Score Analysis

​Key Metrics to Compare

Total Travel Time

Distance Traveled

Planned Deviation

Resource Utilization

​Advanced Evaluation Features

​Evaluating Partial Solutions

​Getting Detailed Explanations

​Best Practices

​Next Steps

Evaluating and Comparing VRP Solutions

Overview of the Evaluation Workflow

Step 1: Generate an Optimized Solution

Step 2: Retrieve the Optimized Solution

Step 3: Create an Alternative Solution

Step 4: Compare the Results

Understanding the Comparison

Score Analysis

Key Metrics to Compare

Advanced Evaluation Features

Evaluating Partial Solutions

Getting Detailed Explanations

Best Practices

Next Steps