Solution Quality Metrics

Understanding solution quality goes beyond simple feasibility. This guide explains the key metrics for evaluating VRP solutions, how to interpret them, and strategies for improvement.

What Makes a Good Solution?

A high-quality VRP solution balances multiple objectives:

Core Quality Metrics

1. Feasibility Rate

The percentage of jobs successfully assigned. The response doesn’t return this rate directly — it only reports which jobs were left out, as unserved: List<String> — so compute it yourself against the job count you sent:

Feasibility Rate = (1 - (unserved.length / jobs.length)) × 100%

Perfect (100%)
Good (95%+)
Poor (<90%)

{
  "unserved": []
}

Ideal scenario - all jobs scheduled.

{
  "unserved": ["job-23", "job-47"]
}

Acceptable with partial planning enabled (2 of 50 jobs unserved = 96%).

{
  "unserved": ["job-3", "job-11", "job-19", "job-24", "job-31", "job-38", "job-44", "job-49"]
}

Indicates capacity or constraint issues (8 of 50 jobs unserved = 84%).

2. Occupancy

How full the schedule is in terms of work time versus available time:

Occupancy = (Total Work Time / Total Available Time) × 100%

Work Time includes:

Travel time
Service time
Wait time (if any)

Available Time:

Shift duration minus breaks

{
  "occupancy": 0.78,  // 78% utilized
  "totalTravelTimeInSeconds": 14400,
  "totalServiceTimeInSeconds": 12600,
  "totalWaitTimeInSeconds": 1080
}

These are flat, top-level fields on the response — there’s no nested breakdown object. Interpreting Occupancy:

< 60%: Underutilized - consider reducing fleet
60-85%: Optimal range - good efficiency with buffer
> 85%: High utilization - limited flexibility
> 95%: Over-stretched - risk of delays

3. Workload Fairness

Measures how evenly work is distributed across resources:

Fairness = 1 - (Standard Deviation / Mean Work Time)

{
  "workloadFairness": 0.85,
  "trips": [
    {"resource": "driver-1", "workTime": 420},
    {"resource": "driver-2", "workTime": 390},
    {"resource": "driver-3", "workTime": 405}
  ]
}

workloadFairness is returned directly by the solver. Per-resource workTime comes from each Trip in the trips array — there’s no separate resourceWorkloads field with a per-resource deviation; sum workTime by resource yourself to compute the average and standard deviation.

4. Travel Efficiency

Multiple metrics evaluate routing efficiency:

Total Distance & Time

{
  "totalTravelDistanceInMeters": 245000,
  "totalTravelTimeInSeconds": 28800,
  "averageSpeed": 30.6  // km/h
}

Distance per Job

Efficiency = Total Distance / Number of Jobs

Lower values indicate better clustering and routing.

Circuity Factor

Circuity = Actual Distance / Direct Distance

Values close to 1.0 indicate direct routes.

5. Time-Based Metrics

On-Time Performance

The response doesn’t return on-time/late delivery counts directly — derive them by comparing each Visit.arrival (in trips[].visits[]) against the corresponding job’s requested windows.

Wait Time Analysis

{
  "totalWaitTimeInSeconds": 3600,
  "jobsWithWaitTime": 8,
  "averageWaitPerJob": 450,
  "waitTimePercentage": 0.05  // 5% of total time
}

High wait times indicate:

Poor time window alignment
Suboptimal route sequencing
Need for dynamic scheduling

Advanced Quality Indicators

Service Level Metrics

None of priority fulfillment rate, customer-preference match rate, or time-window-compliance rate are returned as dedicated response fields. Derive them yourself from what the response does provide:

Priority fulfillment: cross-reference unserved against each job’s priority from your request.
Customer preferences: cross-reference the resource assigned in each Trip/Visit against the job’s rankings.
Time window compliance: compare Visit.arrival against the job’s windows, and inspect unservedReasons for DATE_TIME_WINDOW_CONFLICT / DATE_TIME_WINDOW_CONFLICT_SOFT entries.

Cost Effectiveness

When a costs configuration is provided in the request, the response includes an estimatedCost breakdown:

estimatedCost

{
  "totalCostEur": 450.75,
  "travelTimeCostEur": 125.50,
  "distanceCostEur": 87.50,
  "waitTimeCostEur": 15.00,
  "laborCostEur": 200.00,
  "activationCostEur": 50.00,
  "penaltyCostEur": 27.25
}

There’s no separate utilization-rate, cost-per-job, cost-per-km, overtime-rate, profit-margin, or jobs-per-resource-hour field — derive those yourself from estimatedCost plus your own job/resource counts if you need them.

Comparing Solutions

When evaluating alternative solutions:

Multi-Criteria Comparison

Pareto Optimization

Often no single “best” solution exists. Consider Pareto-optimal solutions:

A solution is Pareto-optimal if improving one metric necessarily worsens another. Keep multiple Pareto-optimal solutions for different scenarios.

Weighted Scoring

Create a composite score based on business priorities:

const weights = {
  feasibility: 0.3,
  efficiency: 0.25,
  cost: 0.25,
  service: 0.2
};

const compositeScore = 
  (feasibilityRate * weights.feasibility) +
  (efficiencyScore * weights.efficiency) +
  (costScore * weights.cost) +
  (serviceScore * weights.service);

Improvement Strategies

For Low Feasibility

Enable Partial Planning

{"options": {"partialPlanning": true}}

Relax Constraints

Convert hard constraints to soft where possible

Add Resources

Increase fleet size or extend shifts

Review Requirements

Check if all constraints are necessary

For Poor Efficiency

Reduce Travel
Minimize Wait
Optimize Routes

{
  "weights": {
    "travelTimeWeight": 5,
    "regionMinimisationWeight": 10
  }
}

{
  "weights": {
    "waitTimeWeight": 20
  },
  "options": {
    "snapUnit": 900  // 15-min slots
  }
}

{
  "options": {
    "euclidian": false,
    "routingEngine": "OSM",
    "traffic": 1.1
  }
}

For Imbalanced Workload

{
  "options": {
    "fairWorkloadPerResource": true,
    "workloadSensitivity": 0.1  // 10% tolerance
  },
  "weights": {
    "workloadSpreadWeight": 100
  }
}

Monitoring Quality Over Time

Track metrics to identify trends:

{
  "date": "2024-03-15",
  "metrics": {
    "feasibilityRate": 0.96,
    "occupancy": 0.78,
    "onTimeRate": 0.94,
    "costPerJob": 42.50
  }
}

{
  "week": "2024-W11",
  "trends": {
    "feasibility": "+2%",
    "efficiency": "-5%",
    "cost": "+3%",
    "satisfaction": "stable"
  }
}

{
  "month": "2024-03",
  "kpis": {
    "avgFeasibility": 0.95,
    "avgOccupancy": 0.76,
    "fleetUtilization": 0.82,
    "customerSatisfaction": 4.2
  }
}

Quality Benchmarks

Industry standards for different sectors:

Sector	Feasibility	Occupancy	On-Time	Fairness
Parcel Delivery	98%+	75-85%	95%+	0.80+
Field Service	95%+	65-75%	90%+	0.85+
Food Delivery	99%+	70-80%	98%+	0.75+
Medical Transport	100%	60-70%	99%+	0.90+

Benchmarks vary by region, business model, and service level agreements. Use these as starting points and adjust based on your specific requirements.

Best Practices

Quality Management Guidelines:

Define Success Metrics: Establish clear KPIs before optimization
Regular Monitoring: Track metrics daily/weekly
Iterative Improvement: Make small, measured changes
Balance Trade-offs: No solution is perfect in all dimensions
Document Decisions: Record why certain trade-offs were made
Benchmark Regularly: Compare against historical performance

Scoring System

Understand score components

Performance Guide

Optimize solution speed

Cost Optimization

Minimize operational costs

Unassigned Reasons

Fix quality issues

Platform

VRP API

Fill API

Solution Quality Metrics

Solution Quality Metrics

What Makes a Good Solution?

Core Quality Metrics

1. Feasibility Rate

2. Occupancy

3. Workload Fairness

4. Travel Efficiency

Total Distance & Time

Distance per Job

Circuity Factor

5. Time-Based Metrics

On-Time Performance

Wait Time Analysis

Advanced Quality Indicators

Service Level Metrics

Cost Effectiveness

Comparing Solutions

Multi-Criteria Comparison

Pareto Optimization

Weighted Scoring

Improvement Strategies

For Low Feasibility

For Poor Efficiency

For Imbalanced Workload

Monitoring Quality Over Time

Quality Benchmarks

Best Practices

Scoring System

Performance Guide

Cost Optimization

Unassigned Reasons

​Solution Quality Metrics

​What Makes a Good Solution?

​Core Quality Metrics

​1. Feasibility Rate

​2. Occupancy

​3. Workload Fairness

​4. Travel Efficiency

​Total Distance & Time

​Distance per Job

​Circuity Factor

​5. Time-Based Metrics

​On-Time Performance

​Wait Time Analysis

​Advanced Quality Indicators

​Service Level Metrics

​Cost Effectiveness

​Comparing Solutions

​Multi-Criteria Comparison

​Pareto Optimization

​Weighted Scoring

​Improvement Strategies

​For Low Feasibility

​For Poor Efficiency

​For Imbalanced Workload

​Monitoring Quality Over Time

​Quality Benchmarks

​Best Practices

​Related Topics

Scoring System

Performance Guide

Cost Optimization

Unassigned Reasons

Solution Quality Metrics

What Makes a Good Solution?

Core Quality Metrics

1. Feasibility Rate

2. Occupancy

3. Workload Fairness

4. Travel Efficiency

Total Distance & Time

Distance per Job

Circuity Factor

5. Time-Based Metrics

On-Time Performance

Wait Time Analysis

Advanced Quality Indicators

Service Level Metrics

Cost Effectiveness

Comparing Solutions

Multi-Criteria Comparison

Pareto Optimization

Weighted Scoring

Improvement Strategies

For Low Feasibility

For Poor Efficiency

For Imbalanced Workload

Monitoring Quality Over Time

Quality Benchmarks

Best Practices

Related Topics