Set of (Benchmark) Problem Instances. â¢ Modeling Language Differences: Few. â¢ Taste (Syntax, Development Environment,â¦) â¢ Platform, Model Type...

0 downloads 0 Views 2MB Size

Model Structure

2

Model Data

3

Matrix Generator

4

Matrix Generator Input

5

MPS File – Column Section

6

MPS Revision File

7

MPS Output

8

WB Old Slide 1

9

WB Old Slide 2

10

WB Old Slide 3

11

WB Old Slide 4

12

WB Old Slide 5

13

WB Old Slide 6

14

WB Old Slide 7

15

Basic Principles

• Separation of model and solution methods •Models is a data base operator and/or object • Balanced mix of declarative and procedural approaches • Computing platform independence •Multiple model types, solvers and platforms 16

Change in Focus Computation Past

Model

Application

Present

Future

•

Algorithm limits applications

•

Modeling skill limits applications

•

Domain expertise limits application

•

Problem representation is low priority

•

Algebraic model representation

•

Off-the-shelf graphical user interfaces

•

Large costly projects

•

Smaller projects

•

•

Long development times

•

Rapid development

Links to other types of models

•

•

Centralized expert groups

Decentralized modeling teams

Models embedded in business applications

•

•

High computational cost, mainframes

Low computational cost, workstations

New computing environments

•

Machine independence

•

Internet/web

Users left out

•

Users involved

•

Users hardly aware of model

•

•

•

17

Dinner Table Assignment • ‘Couples” on the same table • Mix backgrounds and interest (algorithms,applications,World Bank, GAMS staff and family) • Allow for last minute changes • Use a model to shed responsibility for ‘poor’ assignments 18

List of Participants

19

Problem Domains

20

New Excel Input

21

Model Definition

22

Solve Summary

23

Output Report

24

Group Balance

25

Excel Report

26

Data Connectivity • Data Import/Export from Standard Applications – MS Office, Database, Text files, …

• Capture an Instance of a Model Failure – Reproducibility of Model/System Bugs – Problems: Life Database/different Platforms

• Definition of Data Interface – Gams Data eXchange (GDX) – Separation of Responsibility for Data and Model 27

Gams Data eXchange • Gams Data eXchange (GDX): Application

GDX

GAMS

• Complements the ASCII text data input • Advantages: – Fast exchange of data – Syntactical check on data before model starts – Compile-time and Run-time Data Exchange 28

GDX Tools IDE

GDX Viewer

GDX API gams

gdxxrw (MS Office)

gdxsplit

GDX

gdxmerge gdxdump

gdxdiff 29

Data Contract in GAMS

30

Data Contract in Excel

31

Data Contract in Excel – cont.

32

GAMS/MapInfo

Increase in Ktons Per Year Less Than 0 0 - 199 200-1000 1000-3000

33

GAMS/MATLAB

34

35

Three Examples of GDX • VEDA : post processing and ad-hoc information restructuring • LOGMIP: experimental language extensions, algorithmic design studies • Branch & Cut & Heuristic: automation of complex solver enhancements

36

The VErsatile Data Analyst (VEDA) Amit Kanudia, KanORS Consulting Inc. Gary A. Goldstein, IRG 37

VEDA Description • Manages GAMS Sets, Parameters and Model Results for the purpose of dynamic viewing and re-organizing, and presentation to others. • Allows for cross-cutting access to parameters and results according to index structure and element values. • Supports construction of user-defined sets, definition of new tables, and manipulation of tables. • Provides powerful filtering capabilities on both elements names and descriptions. 38

VEDA Description • Employs customizable n-dimensional Data Cube component for displaying tables. • Full-blown graphing facilities, as well as exporting to Excel, Word, PPT, HTML. • Web-enabled, allowing model results to be made available over the web in userfriendly, reconfigurable tables. • Convenient GDX2VEDA data definition file for identification of information to be ported from GAMS to VEDA. 39

VEDA Table Form – Attribute List

40

VEDA Table Form – Sets (Model&User)/Elements

41

VEDA Table Form – Search & Select Engine

42

ProcessSet/Process/Commodity/ Scenario

43

VEDA Cube – Scenario/ProcessSet Totals

44

VEDA Cube – Analysis Graph

45

GDX2VEDA Specification

46

LOGMIP Characterization and Formulation of Disjunctions and their Relaxations

Aldo Vecchietti (*), Sangbum Lee(+) and Ignacio Grossmann (+) (+) Chemical

Engineering Department - Carnegie Mellon University (*) INGAR - Instituto de Desarrollo y Diseño UTN - Facultad Regional Santa Fe

47

Problem Formulations Accepted Formulations

Recent formulations

MILP and MINLP formulations are the most accepted by academia and the industry for problems involving discrete decisions

Constrained Logic Programming (CLP): general logic constraints, applied mainly for highly combinatoric problems Generalized Disjunctive Programming (GDP) : disjunctions and logic propositions for discrete choices

48

Disjunctions min Z = Σ k ck + f(x) + dTy sujeto a: g(x) ≤ 0 r(x) + Dy ≤ 0 Ay ≥ a ∨ i∈Dk

Yik hik (x) ≤ 0 ck = γ ik

k∈ SD

Ω (Y) = True x∈ Rn, y∈ {0,1}q Y∈ {True, False}m , ck ≥ 0

q y binary variables (0-1) q x y ck continuous variables q Yik Boolean variables to establish if a disjunction term is true or false q f(x) objective function (linear or not linear) q dTy Linear cost term q g(x) constraint (linear or not linear) independent of discrete choices q r(x)+Dy≤ 0 mixed integer constraint (linear or not linear) q Ay≥ a integer constraint qΩ (Y) propositional logic relating Boolean variables (disjunction terms) 49

Big-M Relaxation Big-M Linear

[

F = ∨ a Ti x ≤ bi i ∈D

]

Non Linear x ∈ Rn

F = ∨ [hi (x) ≤ 0 ] i∈D

a Ti x ≤ bi + M i (1 - y i )

x ∈ Rn

hi(x)≤ Mi(1-yi)

∑ yi = 1

∑ yi = 1

i

i

Mi = max{ aiT x – bi | xlo≤ x≤ xup}

Mi = max{ hi(x) | xlo≤ x≤ xup}

Beaumont Surrogate

a Ti x b ≤ ∑ i + N -1 ∑ i Mi i Mi

∑ i

hi ( x ) ≤N -1 Mi 50

Convex Hull Relaxation Linear

[

F = ∨ a Ti x ≤ bi i ∈D

x - ∑ vi = 0 i ∈D

Non linear

]

x ∈ Rn

x, v i ∈ R n

F = ∨ [hi (x) ≤ 0 ] i∈D

x - ∑ vi = 0 i∈ D

a Ti v i - bi yi ≤ 0

y i hi ( v i / yi ) ≤ 0

∑ yi = 1

∑ yi = 1

i∈D

, 0 ≤ y i ≤ 1, i ∈ D

0 ≤ v i ≤ v iup y i

i∈ D

x ∈ Rn

x, v i ∈ R n

, 0 ≤ y i ≤ 1, i ∈ D

0 ≤ v i ≤ v iup y i 51

Relaxation Comparison Improper disjunction of special interest in Process Engineering

X2

1

The x space

minZ = ( x1 − 1.1 )2 + ( x2 − 1.1 )2 + c1 s .t . Y1 ¬Y1 2 2 x + x ≤ 1 2 1 ∨ x1 = x2 = 0 c =1 c = 0 1 1 0 ≤ x1 , x 2 ≤ 1; 0 ≤ c1

1

X1

Both relaxations have the same feasible region

Y1 ∈ { true , false } 52

X – Y Space Convex hull

y

y

Big-M

X1

X1 X2

minZ = ( x1 − 1.1 )2 + ( x 2 − 1.1 )2 + y1 s .t . x12 + x 22 ≤ y12 0 ≤ x1 ≤ y1 0 ≤ x 2 ≤ y1 0 ≤ y1 ≤ 1

X2

minZ = ( x1 − 1.1 )2 + ( x2 − 1.1 )2 + y1 s .t . x12 + x22 ≤ y1 0 ≤ x1 , x 2 ≤ 1; 0 ≤ y 1 ≤ 1 53

Two Term Disjunction

False True Constraint 1 ∨ Constraint

Modeling two terms disjunction 2

IF (condition1) THEN Constraints to be considered when condition1 is True ELSE Constraints to be considered when condition1 is False END IF

54

N Term Disjunction 1 2 Constraint s 1 ∨ Constraints 2 ∨ .. ∨

N Constraint s N

IF (condition1) THEN Constraints to be considered when condition1 is True ELSIF (condition2) THEN Constraints to be considered when condition1 is True ELSIF (condition3) THEN ... ELSIF (conditionN) THEN Constraints to be considered when condition1 is True END IF 55

Example min c + 2 x

+x

1

2

Y Y − x + x + 2 ≤ 0 ∨ 2 − x ≤ 0 c=5 c = 7 Y ¬Y x − x ≤ 1 ∨ x = 0 Y ∧ ¬Y ⇒ ¬Y 1

1

2

2

2

3

1

3

2

1

1

2

3

¬(Y ∧ Y ) 2

3

0 ≤ x ≤ 5 , 0 ≤ x ≤ 5, c ≥ 0 1

2

Y ∈ {true, false}, j = 1,2,3. j

56

GAMS Implementation $ONTEXT BEGIN LOGMIP EQUAT1.. X('2')X('2') - X('1') + 2 =L= 0;

IF Y('1') THEN EQUAT1;

EQUAT2.. C =E= 5;

EQUAT2;

EQUAT3.. 2 - X('2') =L= 0; EQUAT4.. C =E= 7;

ELSIF Y('2') THEN EQUAT3;

EQUAT5.. X('1')X('1') -X('2') =L= 1;

EQUAT4;

EQUAT6.. X('1') =E= 0; INT1..

Y('1')+ Y('3') =L= 1;

INT2..

Y('2')+ (1(1-Y('3')) =G= 1;

INT3..

Y('2')+ Y('3') =L= 1;

FICT.. SUM(I, Y(I)) =G= 0;

ENDIF; IF Y('3') THEN EQUAT5; ELSE EQUAT6;

OBJECTIVE.. Z =E= C + 2*X('1') + X('2');

ENDIF;

X.UP(J)=20;

$OFFTEXT END LOGMIP

C.UP=7;

OPTION MIP=LOGMIPC; MODEL PEQUE2 /ALL/; SOLVE PEQUE2 USING MIP MINIMIZING Z; 57

Branch-and-Cut & Heuristics in GAMS for MIP Problems Hua (Edward) Ni George Washington University

Branch-and-Cut (B&C) • B&C is an established algorithm to improve the B&B search. • Implementation facilities: – MIP solver callback functions (CPLEX, XPRESS, …) – B&C framework (ABACUS, COIN BCP, …)

• Required Knowledge for B&C – IT knowledge (C/C++/JAVA, Solver APIs) – Mathematical programming knowledge – Application specific knowledge

• Supply GAMS users with an easy access to B&C 59

Design Principle GAMS SYSTEM

MIP Solver (e.g. CPLEX)

B&B Callback

GAMS Solver Link

User Cut Generator & Heuristics (e.g. GAMS)

A Steiner Tree Problem • Berlin52 – from SteinLib – 52 nodes (1 source, n’=15 sinks), 1326 edges

• Flow formulation: min

∑f

(i , j )∈A

ij

yij , s.t.

• Dicut: • Separation:

∑x

ji ( j ,i )∈δ − ( i )

−

∑x

ij ( i , j )∈δ + ( i )

= bi ,0 ≤ xij ≤ n ' yij , y ∈{0,1}

if S⊂V and b(S)>0.

Cut Generator in GAMS I

62

Cut Generator in GAMS II

63

Computational Results • Overhead – Time spent within the callback functions minus MIP computation on cuts and heuristics. – 20% ~ 25%

• Performance Improvements (B&C vs. regular GAMS/CPLEX) – Steiner: 6 hours vs. 2+ days (unsolvable) – Pipeline Design: 20 minutes vs. 450 minutes

Convergence – Steiner Tree 1252.8

1044

B&C&H_Dual B&C&H_Primal B&C_Dual B&C_Primal CPLEX_Dual

835.2

CPLEX_Primal

626.4 0

300

600

900

1200

1500

1800

Compuation Time (seconds)

65

Convergence – Pipeline Design 1636.45

1565.3

B&C&H_Dual

1494.15

B&C&H_Primal CPLEX_Dual CPLEX_Primal B&C_Dual

1423

B&C_Primal

1351.85

1280.7 0

300

600

900

1200

1500

1800

Computation Time (seconds)

66

What is a Model? • List of Equations – Mathematical Programming (MP) Model

• Collection of several intertwined (MP) Models – Data Preparation and Calibration – “Solution” Module – Reporting Module

• Categorization of Models by answering: – Who is the User of a Model? 67

Who is the User of a Model? • (Academic) Researcher – One time use (Research Paper)

• Domain&Model Expert – Model Results used for Consulting

• Black Box User – Model integrated in (Optimization) Application

• Each Category has its own needs – Development & Deployment 68

Research Models • • • • • •

95% of Model Source is (Equation) Algebra Declarative Modeling Set of (Benchmark) Problem Instances Modeling Language Differences: Few Taste (Syntax, Development Environment,…) Platform, Model Type, Solver 69

Supported Solvers

70

Available Solvers • Growing number of MP Solvers (often out of Academic Labs) • NEOS: >40 Solvers • Impractical to have Interface to all Modeling Languages • “Solution”: Model Translation 71

Model Translation

72

Consulting Models • Model is Tool for Problem Analysis • 10% of Model Source is (Equation) Algebra • User: Domain & Modeling Expert (not necessary the same person) • Living Model (changes with the problem) – Lifecycle: At least 10 years – Technology Change (Platform, Solver, …) 73

Figure I-1

Hill and Associates, Inc. Electric Generation, Coal and Emissions Forecasting System Generation Database (all units)

Generation Cost Supply Models • New Build • Gas & Oil Forecast

• Unit Info • Transp. Rates • Compliance Costs

Strategies for • SO2 • NOx • Particulates

Seasonal Dispatch Costs, Emission Rates

Coal Plant Costs &Emissions Forecast

National Power Model™ • Dispatch Economics • 90+ Control Areas • Seasonal/TOD Prices, Flow & Gen.

Plant/Area by TOD • Generation • Power Flows • Marginal Prices • Emissions

Demand Model

GDP Weather Electric Intensity

Regional Emission Limits • SO2, CO2, NOx Transmission • Bi-Directional Simultaneous Flows • Seasonal Limits • Time-of-Day Rates

Coal Plant Energy Demand

Generation Database (coal units)

Generation Compliance Models

Utility Fuel Economics Model • Fuel Switching • Clean-up Equip Choices • Allowance Trading

Coal Supply Curves • Cash Cost by Mine • All Regions • By Coal Type

International Coal Trade Model Coal Marginal Cost Models

I-2

Demand • Industrial • Commercial • Residential

Modeling System Requirements • Survive in such a diverse Environment • Compatibility – 15 year application lifecycle

• Data Connectivity/Exchange – Programs and People

• Support for Analysis/Reporting – Modeling System Tools and external Program 75

Black Box Model • Innocent User • Bulletproof Optimization Application – No failures: e.g. No Infeasible Models

• Model embedded in larger System: • Optimization • Takes Longer than one is willing to wait • It will eventually fail

• Application • Real Time • Always need a Solution to Problem 76

Scheduling US Military Academy West Point “ … each student’s daily activities are a carefully regimented balance of academic, military, and physical requirements.”

77

Modeling System Requirements • Reliable System • Using Third Party Software (Solver) – Keep Resource Usage of Solver in check – Solver will eventually crash

• Possibility to Implement Simple Heuristics • Platform Choice • Less important: Data Import/Export – IT does not want Modeling System to mess with DB – Simple/Thin Interfaces (text files, XML) 78

Future Directions • • • • •

Value Added Applications Solution Service Providers Distributed System Architectures New Solution Approaches Continued Changes in the Modeling ‘Industry’ 79

Our partners will collect data and use cookies for ad personalization and measurement. Learn how we and our ad partner Google, collect and use data. Agree & close