Programming is a dynamic discipline. It is likely that also in the scientific computing community, the approach to programming will continue to evolve in the decades to follow.
How to extrapolate the future?
When viewed in terms of programming paradigms and tools, because computational scientists are not software engineers, in many areas the computational science community tends to trail behind the mainstream programming community. Roughly speaking, object-oriented programming, which has been mainstream in the programming community since the 1980s-90s, has found its way into scientific codes during the last 10-20 years.
(There are notable exceptions where the computational science community is the better informed one, such as parallelization, and the fine details of floating point numbers, since these are highly relevant to computing correct answers quickly.)
Right now, functional programming (FP) is trending toward mainstream. Although strictly speaking, the following only amounts to anecdotal evidence, the signs seem clear: there is the popularity of Haskell in academic circles, as well as the recent rise of functionally flavored languages running on top of the Java VM, such as Clojure, Scala and Kotlin. Even Java 8 itself contains some FP features (see e.g. these motivating posts [1] [2]).
Important motivations for FP are improved modularity (Hughes, 1984), and robustness. As Abelson and Sussman argument in their classic textbook Structure and Interpretation of Computer Programs, mutable state - which is extensively used in the classical imperative programming paradigm - makes the behavior of programs substantially harder to analyze, and introduces whole new categories of possible mistakes, which can be eliminated by favoring an FP approach.
From these observations: it is likely that as tools improve, FP will be mainstream also in scientific codes in (at most) 20-30 years from now.
The future is not yet, so the tools are not yet there; right now, Python is the practical option. Nevertheless, the appropriate time to become familiar with the basic ideas of FP is now; to walk among the pioneers, as well as to be prepared to easily learn more later.
It is still difficult to say which particular school of thought, and which programming language, will eventually become dominant. Haskell? Some member of the Lisp family? Something completely unexpected?
Haskell is a pure functional programming language with strong roots in mathematics (category theory), and comes with a highly advanced static type system. Because no real-world program is completely free of side effects, these are hidden inside monads, a central concept in Haskell that does not appear in most other programming languages.
For our purposes of looking beyond Python, we will concentrate on the other main option - impure FP, working in the Lisp family of programming languages. Some of these techniques can be imported (or even imported) into Python; see lecture 11.
For a mathematician, a monad is a monoid in the category of endofunctors, but maybe that does not help much.
For a programmer, a monad is a design pattern that generalizes function composition (see lecture 10, slide 12), helping to simplify code and eliminate lots of boilerplate.
See e.g. these useful practical explanations:
- You Could Have Invented Monads! (And Maybe You Already Have.),
- Monads, part 1: a design pattern (in Python, no less!), and
- "What is a monad?" on StackOverflow.
If you would like to use monads in Racket, see the library Monads in Racket, and the explanatory blog post from the author, Monads in Dynamically-Typed Languages.
If you would like to use monads in Python, see e.g.:
- PyMonad,
- OSlash,
- Monad-Python (with a nice application to computing probability distributions; see test3.py) and
- Monads in Python.
In short, Lisp is a family of programmable programming languages (as pointed out by John Foderaro) that began in 1958, one year later than Fortran 1. The original language was called LISP (LISt Processor). It is the second oldest high-level language whose direct descendants are still in active use.
Keep in mind that high-level means anything above assembly; Lisp, just like Python, operates at a much higher level of abstraction than e.g. Fortran or C.
While Python can be thought of as executable pseudocode, Lisp has been called executable mathematics (L. Peter Deutsch). (Admittedly, at the time of this writing, Haskell also has a valid claim to that title, although for different reasons.)
The distinguishing feature of Lisps is that the syntax is user-modifiable, via a feature called syntactic macros. This is much more powerful than, and should not be confused with, C preprocessor macros. Whereas the C preprocessor performs string substitution, syntactic macros manipulate the AST by running arbitrary code. The metalanguage for writing macros is (all of!) Lisp itself; contrast generics in C++, where a separate templating mini-language is used.
Hygienic macros are a notable improvement over the early days of LISP, avoiding the issues of identifier capture and free symbol capture (see [1] [2]). Racket adds a tower of phase levels; see this short explanation and the paper by Flatt (2002). For an introduction to Racket's (highly advanced) macro system, see Fear of Macros and The Racket Guide.
Macros can be used to create abstractions for design patterns that cannot be extracted as functions. Furthermore, macros allow the user to add language features that, in most languages, require the language designer's approval, a standardization process, and the distribution of a new language version. For example, Python added a dedicated matrix multiplication operator in Python 3.5; whereas in Lisps, a matrix library can add that, requiring no changes to the core language.
In fact, the core of many Lisps does not even include a for loop - it is not necessary, because iteration is equivalent to tail recursion, and macros can be used to add a for syntax element if desired. Of course, the standard library covers this; the point is that for is implemented in the library, not in the language itself.
Macros facilitate a programming approach somewhat unique to the Lisp community - bottom-up programming, i.e. bringing the language iteratively closer to the problem domain, until the concepts are at a level that is convenient to use to express the solution. See also extensible programming, language-oriented programming, and Felleisen et al. (2018).
A word of caution when reading about Lisp online: contrary to popular belief, it can be shown (Felleisen, 1991) that Lisp is not an asymptotic limit of programming languages that everything in the Fortran branch is converging toward. As an important example, first-class lazy functions cannot be implemented in a language that only offers eager (strict) evaluation, such as most Lisps. In an eager language it is possible to implement lazy evaluation via a macro (e.g. Racket does so), but implementation of this feature via a regular function is not possible. (There is lazy/racket, but that is a separate language.)
(See Evaluation strategy in Wikipedia for more. See also family tree of programming languages.)
Generally speaking, each programming language has its strengths and weaknesses. The strength of Lisp is to be able to modify the language to suit one's needs. Some nontrivial examples include:
- Implementing return, while, and exception handling using call/cc (see also continuations; Racket docs) and macros
- Modifying the reader layer to use indentation instead of parentheses (like in Python) (Racket has a package for this)
- User-programmable infix operators
- Non-deterministic evaluation, enumerating through answers that satisfy a given criterion
- Monads
For the purposes of this brief exposition of features beyond what Python offers, we have chosen the Lisp family, and specifically Racket.
How trying to write Lisp code in Python looks like: Python Obfuscation #1 (Thomas Baruchel).
In the same spirit,
f = lambda lst: (lambda seen: [seen.add(x) or x for x in lst if x not in seen])(set())
L = [1, 1, 3, 1, 3, 2, 3, 2, 2, 2, 4, 4, 1, 2, 3]
print(f(L))because Python has no let. If it did, we could de-obfuscate this to:
f = lambda lst:
let [seen = set()]:
[seen.add(x) or x for x in lst if x not in seen]
L = [1, 1, 3, 1, 3, 2, 3, 2, 2, 2, 4, 4, 1, 2, 3]
print(f(L))Finally, for an elaborate piece of programming humor, see Statementless Python (which may start to sound vaguely familiar roughly one third the way through).