Performance measurements

Collecting sensible performance numbers require a lot of care. There are several reasons the numbers may change from one measurement to another. For example, paging and caching affect the speed of data fetching, adaptive CPU clock rates, interaction of thread scheduling with pipelining, and ordering of benchmarks may change the numbers. So one should time the same thing in many different ways, paying attention to small differences between each run.

Comparing the performance of your code to that of library functions like List.map can be tricky since in the latter, cross-module optimizations are hindered. In the spirit of scientific experimentation, we should keep the differences between timed pieces of code to a bare minimum. For library functions, it's best to also write your own version locally to compare against.

For most functions we wrote on lists, we expect the performance on sized and unsized lists to be the same. One exception is the zip function. For zip, we expect that there's less checking in the indexed type.

Stack size, CPS

Even for relatively small sizes, the list functions are found to run out of stack space. The recommended workaround now for it is to increase the stack size. Another way of dealing with this is to convert to continuation passing style (CPS), and we'll look at this later on in the course (not now).

Another question that came up in this context was whether indexed types affect what we can do with CPS. Basically, Hindley-Milner types are enough to let us do most of the main things that we would want to do with CPS. There are a few interesting things we can do with indexed types, but we won't worry about them in this seminar (see for example Hayo Thielecke's works on continuations).

Yet another thing that came up was that Joseph mentioned the Typeful Programming paper and he wasn't satisfied with using int in place of nat, since the former has negative numbers. So the question was, what the best way is to deal with this mismatch. Because the focus of the seminar is to learn how to work with the more expressive types that Concoqtion has, the recommended way to program with nat is to define an OCaml type for naturals. This ensures that we have a much better match between the the values we deal with and the proofs that we do about them at the level of types.

Very often, efficient data structures are more complicated and difficult to reason about. We'll eventually want to look at such efficient data structures but initially, we'll only focus on simple ones. For example, the int type in OCaml is really a ring, which is much more involved to reason about than an inductively defined type, nat. ints are of course much more efficient but we'll focus on nats for now.

Problem with OCaml (non-dependently typed) match

Using OCaml match on dependently typed programs causes problems, so we should try to avoid that. A case in point, Cherif had the following example:

match size_it blah with | E .|m'(nat)| ls -> ls

gives rise to a list containing a free kind variable (UNBOUND_REL)in its type. This doesn't have an "in" specification (so it's an OCaml match, not a Concoqtion match) so a kind variable shouldn't be allowed to appear explicitly or implicitly in the return type. The type checker misses that.