Building Pythonic MIPs with AMPL

Update

Sadly, as of June 2021, AMPL has failed to follow through on their early promise of connecting their modeling language to Python in a professional way. Unlike gurobipy, the AMPL licensing and digital rights management remains quite primitive. Thus, one of the main advantages of Python, code portability, is lost when using amplpy. With gurobipy, licensing is separated from individual machines via the Gurobi Cloud Platform. A gurobipy based package can simply be deployed with a gurobi.lic that points to a Gurobi cloud account. As amplpy lacks similar functionality, an AMPL customer is unable to distribute their licensing agreement with an amplpy based package. So long as AMPL licensing remains mired in the last century, I can't with good conscience recommend AMPL based Python development. The post I originally wrote a few years ago was based on an understanding that AMPL would upgrade their licensing to be on par with that of Gurobi, but all evidence is that AMPL is unable or unwilling to make this step. Hopefully they will do so at some time soon, as I do think their original efforts with amplpy showed real promise.

Original article below

It's possible I've spent more time pondering the combination of optimization modeling languages and Python than any person in history. I'm not suggesting all this contemplation has been productive, or that I'm the leading expert on the subject. (There are at least two gear-heads ahead of me just counting Gurobi employees). Regardless, the dilemma of reckoning the well established modeling language community and the emerging data science standardization around Python certainly does occupy a large part of what might appear to be idle thought.

The crux of this dilemma rests on the following two assertions - large, well respected communities will always demand accommodation, and a declarative language will always render a system of equations more naturally than an imperative language. The former is simply a fact of history, and doesn't really need analysis. Modeling languages (particularly AMPL) are deeply entrenched in the academic community. Unless Python discovers a way to respect this choice, it's appeal and adoption will suffer.

The latter is an opinion I believe, even though it doesn't reflect my practice. I have spent enough time building MIPs with Python and gurobipy that the "Python-way" of writing summations, for-each and for-all feels natural to me. But that is a bit like preferring Reverse Polish Notation calculators to "normal" calculators. Nobody (other than Dijkstra, pictured below) starts out preferring post-fix to algebraic notation.

Instead, you train your brain to think in the less-natural way, because you believe it will eventually enable you to solve those problems more efficiently. That said, if the objective is to resemble LaTeX, then a language that can support declarative idioms has a huge head start . As a result, such a solution will inevitably be more attractive than the declarative syntax of Python to many (perhaps even most) practitioners for rendering equations.

That said, arguments along the lines of "pure-Python or AMPL-embedded-Python" would miss the point. My sense is that someone beginning Python with pre-existing AMPL experience would likely prefer the latter, and someone who is fluent with Python comprehensions the former. But I would encourage anyone who is serious about implementing optimization engines to eventually master both techniques. This is certainly my intention.

I've recently had the pleasure of exploring this topic in greater detail, as I developed Python and AMPL example engines using the amplpy package. This package is available on GitHub and pypi and distributed under an open source license. You need to supplement your pip installation with a download from AMPL in order to exercise the full range of routines.

I was very impressed with what AMPL has accomplished with amplpy. My experience with Python and AMPL was completely different than my experience with Python and OPL. For the latter, there is no Python package, and a Python developer really is required to develop a fairly sophisticated series of subroutines in order to combine the two languages. (I blogged about these subroutines here). Moreover, even with such routines in hand, you still have the general awkwardness of calling a language via a subprocess command instead of a true Python API. This is more than just aesthetics. For example, graceful exception handling is often a requirement for a robust application, and for that you need something equivalent to amplpy.

As much as I liked amplpy, I still felt like there were ways in which I could usefully extend it. To this end, I developed a series of amplpy convenience routines within the ticdat package. These ticdat routines simplify the process of copying input data into an amplpy.AMPL object, and copying solution data out of this same object after a successful solve. I also created an AMPL friendly variation of the standard Python string.format routine. The input routines are copy_to_ampl and set_ampl_data, the output routine is copy_from_ampl_variables and the formatting routine is ampl_format. You can read more about them from the collected docstrings here and the examples I built here.

At the risk of sounding self-congratulatory, I feel like the work I have done with ticdat brings amplpy closer to the core values of AMPL. Specifically, one of AMPL's great innovations was the clear delineation of .mod files for model logic and .dat files for data handling. The ticdat package is focused exclusively on providing .dat type functional for math engines built with Python. The diet.py, netflow.py, metrorail.py example files combine .mod code and Swiss-Army knife .dat functionality into one coherent file.

I include the data and modeling logic in the same file only because the models are so small. Python makes it very easy to split package code across .py files in whatever manner you see fit, and for richer examples the .mod logic would likely merit a dedicated file. But even combined in one file, I find the data/modeling separation to be just as clear as with AMPL's original .mod, .dat methodology.

As always, ticdat is an open-source package whose functionality can be exercised fully without any additional Opalytics licensing. I hope that my work with ticdat and amplpy can help accelerate the adoption of Python by the OR community. Regardless of whether you use ticdat directly, or simply draw inspiration to write your own personalized convenience routines, I think you will agree that amplpy is a useful addition to your Python skill-set.