One of the first challenges in the HackerRank‘s “Warmup” section is probably the “Hello World” of learning arrays in any language: calculating the sum of a sequence of elements. Although this exercise is trivial, I’ll face with it to break the ice and show you a few concepts that lay the groundwork for more complicated challenges.

I’m assuming you are already familiar with concepts like iterator, container and algorithm. Most of the time I’ll give hints for using these C++ tools effectively in Competitive Programming.

That’s the problem specification: You are given an array of integers of size N. Can you find the sum of the elements in the array? It’s guaranteed the sum won’t overflow the int32 representation.

First of all, we need an “array of size N”, where N is given at runtime. The C++ STL (Standard Template Library) provides many useful and cleverly designed data structures (containers) we don’t need to reinvent. Sometimes more complicated challenges require us to write them from scratch. Advanced exercises reveal less common data structures that cannot be light-heartedly included into the STL. We’ll deal with some examples in the future.

It’s not our case here. The primary lesson of this post is: don’t reinvent the wheel. Many times standard containers fit your needs, especially the simplest one: std::vector, basically a dynamic sequence of contiguous elements:

For the purpose of this basic post, here is a list of important things to remember about std::vector:

  • it’s guaranteed to store elements contiguously, so our cache will love it;
  • elements can be accessed through iterators, using offsets on regular pointers to elements, using the subscript operator (e.g. v[index]) and with convenient member functions (e.g. at, front, back);
  • it manages its size automatically: it can enlarge as needed. The real capacity of the vector is usually different from its length (size, in STL speaking);
  • enlarging that capacity can be done explicitly by using reserve member function, that is the standard way to gently order to the vector: “get ready for accommodating N elements”;
  • adding a new element at the end of the vector (push_back/emplace_back) may not cause relocation as far as the internal storage can accommodate this extra element (that is: vector.size() + 1 <= vector.capacity());
  • on the other hand, adding (not overwriting) an entry to any other position requires to relocate the vector (eventually in the same block of memory, if the capacity allows that), since the contiguity has to be guaranteed;
  • the previous point means that inserting an element at the end is generally faster than inserting it at any other position (for this reason std::vector provides push_back, emplace_back and pop_back member functions);
  • knowing in advance the number of elements to store is an information that can be exploited by applying the synergic duo reserve + push_back (or emplace_back).

The latter point leads to an important pattern: inserting at the end is O(1) as far as the vector capacity can accommodate the extra element – vector.size() + 1 <= vector.capacity(). You may ask: why not enlarging the vector first and then just assign values? We can do that by calling resize:

resize enlarges the vector up to N elements. The new elements must be initialized to some value, or to the default one – as in this case. This additional work does not matter in this challenge, however initialization may – in general – cause some overhead (read, for example, these thoughts by Thomas Young). As a reader pointed out on reddit, push_back hides a branching logic that can cause some cost. For this reason he suggests that two sequential passes over the data (that is contigous) may be faster. I think this can be true especially for small data, however the classical recommendation is to profile your code in case of such questions. In my opinion getting into the habit of using reserve + *_back is better and potentially faster in general cases.

The heart of the matter is: need a dynamic array? Consider std::vector. In competitive programming std::vector is 99% of the time the best replacement for a dynamic C-like array (e.g. T* or T**). 1% is due to more advanced challenges requiring us to design different kind of dynamic arrays that break some std::vector’s guarantees to gain some domain-specific performance. Replacing std::vector with custom optimized containers is more common in real-life code (to have an idea, give a look for example here, here and here).

If N was given at compile-time, a static array could be better (as far as N is small – say less than one million – otherwise we get a stack overflow). For this purpose, std::array is our friend – basically a richer replacement of T[]. “Richer replacement” means that std::array is the STL-adaptation of a C-array. It provides member functions we generally find in STL containers like .size(), .at(), .begin()/.end(). std::array combines the performance and accessibility of a C-style array with the benefits of a standard container. Just use it.

Since much information is stated in the problem’s requirements, we’ll see that static-sized arrays are extraordinarily useful in competitive programming. In the future I’ll spend some time about this topic.

Now, let’s look at my snippet again: can we do better? Of course we can (from my previous post):

At this point we have the vector filled and we need to compute the sum of the elements. A hand-made for loop could do that:

Can we do better?

Sure, by using the first numeric algorithm of this series: ladies and gentlemen, please welcome std::accumulate:

One of the most important loops in programming is one that adds a range of things together. This abstraction is known as reduction or fold. In C++, reduction is mimicked by std::accumulate. Basically, it accumulates elements from left to right by applying a binary operation:

accumulate with three parameters uses operator+ as binary operation.

std::accumulate guarantees:

  • the order of evaluation is left to right (known also as left fold), and
  • the time complexity is linear in the length of the range, and
  • if the range is empty, the initial value is returned (that’s why we have to provide one).

The reduction function appears in this idiomatic form:

So the result type may be different from the underlying type of the range (ElementType). For example, given a vector of const char*, here is a simple way to calculate the length of the longest string by using std::accumulate (credits to Davide Di Gennaro for having suggested this example):

To accumulate from the right (known as right fold) we just us reverse iterators:

Right fold makes some difference – for example – when a non-associative function (e.g. subtraction) is used.

In functional programming fold is very generic and can be used to implement other operations. In this great article, Paul Keir describes how to get the equivalent results in C++ by accommodating std::accumulate.

Does std::accumulate have any pitfalls? There exist cases where a+=b is better than a = a + b (the latter is what std::accumulate does in the for loop). Although hacks are doable, I think if you fall into such scenarios, a for loop would be the simplest and the most effective option.

Here is another example of using std::accumulate to multiply the elements of a sequence:

std::multiplies<> is a standard function object (find others here).

Using standard function objects makes the usage of algorithms more succinct. For example, the problem of finding the missing number from an array of integers states: given an array of N integers called “baseline” and another array of N-1 integers called “actual”, find the number that exists in “baseline” but not in “actual”. Duplicates may exist. (this problem is a generalization of the “find the missing number” problem, where the first array is actually a range from 0 to N and a clever solution is to apply the famous Gauss’ formula N(N+1)/2 and subtracting this value from the sum of the elements “actual”). An example:

The missing number is 2.

A simple linear solution is calculating the sum of both the sequences and then subtracting the results. This way we obtain the missing number. This solution may easily result in integer overflow, that is undefined behavior in C++. Another wiser solution consists in xor-ing the elements of both the arrays and then xoring the results.

Xor is a bitwise operation – it does not “need” new bits – and then it never overflows. To realize how this solution works, remember how the xor works:

Suppose that “a” is the result of xor-ing all the elements but the missing one – basically it’s the result of xor-ing “actual”. Call the missing number “b”. This means that xor-ing “a” with the missing element “b” results in xor-ing together the elements in the “baseline” array. Call the total “c”. We have all the information to find the missing value since “a” ^ “c” is “b”, that is just the missing number. That’s the corresponding succint C++ code:

Let’s go back to the initial challenge. We can do even better.

To realize how, it’s important to get into the habit of thinking in terms of iterators rather than containers. Since standard algorithms work on ranges (pairs of iterators), we don’t need to store input elements into the vector at all:

Advancing by one – using next – is a licit action since the problem rigorously describes what the input looks like. This snippet solves the challenge in a single line, in O(n) time and O(1) space. That’s pretty good. It’s also our first optimization (actually not required) since our solution dropped to O(1) space – using std::vector was O(n).

That’s an example of what I named “standard reasoning” in the introduction of this series. Thinking in terms of standard things like iterators – objects making algorithms separated from containers – is convenient and it should become a habit. Although it seems counterintuitive, from our perspective of C++ coders thinking in terms of iterators is not possible without knowing containers. For example we’ll never use std::find on std::map, instead we’ll call the member function std::map.find(), and the reason is that we know how std::map works. In a future post I’ll show you other examples about this sensible topic.

Our solution leads to ranges naturally:

view::tail takes all the elements starting from the second (again, I skipped the input length), and ranges::istream is a convenient function which generates a range from an input stream (istream_range). If we had needed to skip more elements at the beginning, we would have used view::drop, which removes the first N elements from the front of a source range.

Iterators-based and ranges-based solutions version look very similar, however – as I said in the introduction of this series – iterators are not easy to compose whereas ranges are composable by design. In the future we’ll see examples of solutions that look extremely different because of this fact.

In Competitive Programming these single-pass algorithms are really useful. The STL provides several single-pass algorithms for accomplishing tasks like finding an element in a range, counting occurrences, veryfing some conditions, etc. We’ll see other applications in this series.

In the next post we’ll meet another essential container – std::string – and we’ll see other algorithms.

Recap for the pragmatic C++ competitive coder:

  • Don’t reinvent containers whenever standard ones fit your needs:
    • Dynamic array? (e.g. int*) Consider std::vector
    • Static array? (e.g. int[]) Use std::array
  • Prefer standard algorithms to hand-made for loops:
    • often more efficient, more correct and consistent,
    • more maintainable (a “language in the language”)
    • use standard function objects when possible
    • use std::accumulate to combine a range of things together
    • If customizing an algorithm results in complicated code, write a loop instead
  • Think in terms of standard things:
    • iterators separate algorithms from containers
    • understand containers member functions

Welcome back to my series about Competitive Programming. Here is the introduction in case you missed it.

In this post I’ll explain some common idioms to deal with input and output.

In C++ a simple task like reading an integer from the standard input can be done in different ways: using streams, C functions or OS-dependant calls. The streams model offers a pretty high level interface, but it is generally slower than using native operating system calls. However, in different cases, it is acceptable.

I have solved a lot of challenges and very rarely I had to switch to C functions (e.g. scanf) or turn off the synchronization between C++ streams and standard C streams after each input/output operation (by using std::ios_base::sync_with_stdio). Most of the time the I/O is not the point of the exercise then we can use the convenient streams model. This point seems irrelevant but it brings about simplification, enabling us to write not only simpler but also safer idioms. We’ll see that in some cases the streams interface is enough for basic parsing as well.

Due to the nature of the challenges – generally not focusing on I/O – I remark that the following idioms are pragmatic: they “work here”, but they may not fit a production environment where streams are often avoided like the plague. As I have read in a comment on reddit: “I fear that it may lead some people to prefer quick and functional solutions that work for some cases, neglecting important traits such as scalability, performance or cross-platform compatibility”. I think that solving challenges is about prototyping a solution which has to cover some sensible scenarios cleverly set up by challenge moderators. “Production” is another business. Definitely. Requirements evolve, scalability matters, etc. Being well-versed in Competitive Programming is not enough, sure, but I think that solving challenges is an entertaining way to face new problems that maybe – at some point – you’ll deal with for “real life purposes”.

I repeat a point from the introduction that I consider the most important: competitive programming is guaranteed brain excercise: “on any given day, what you’re asked to do at work may or may not be challenging, or you may be exercising one of the many non-technical skills required of software developers. By practicing regularly with competitive programming problems, you can ensure that the coding part of your brain receives a regular workout. It’s like one of those brain training apps, but for real skills”. In addition to this, other eleven reasons are wisely explained in this nice article which I recommend again.

After this short additional introduction, let me show some code about I/O.

Just for completeness, in the simplest cases you only need to read lonely values like numbers or strings:

int N, M;
cin >> N >> M;

Remember in these challenges you don’t need to validate the input (unless stated otherwise – never found).

In one of the most common scenarios you need to read a sequence of numbers, and you can do it by using the synergetic copy_n + istream_iterator + back_inserter trio. That’s our first idiom. Generally the length of the sequence is passed just before (and the numbers are separated by whitespaces):

reserve prepares the vector to host “length” elements – it’s possibly allocating memory. A note: I’d really prefer writing something like vector<int> sequence(reserve, length) where “reserve” here is just a tag. The same applies for resize, etc. And this would avoid initializer lists overload predominance:

I used copy_n instead of copy because not only is it clearer, but also convenient in case of more input to read (if I used copy then I would need to pass an end-of-stream iterator, but passing istream_iterator<int>() is dangerous because copy goes on iterating until the stream gets invalid).

With ranges the code streamlines:

bounded returns a range of exactly length elements, in this case from the standard input (view::take is another possibility).

Since a default operator>> for vector is not defined, reading a matrix needs manual iteration (here using copy_n is even more convenient):

Remember we don’t want to create a library, because it couldn’t be used to solve challenges. For this reason we don’t introduce an operator>> for vector.

Other cases involve strings, not just words but also lines. Pay attention to getline: as I have recently blogged (for another reason, but the lesson learned was the same), getline is an unformatted function. This means it does not ignore leading delimiters (newline by default). These innocent lines can lead to something you may don’t expect:

int N; string line; 
cin >> N; getline(cin, line);

The problem here is that we want to ignore the separator between the int and the line. E.g.:

10'\n'
a line representing some data

Since getline does not skip the leading ‘\n’, it will stop immediately, writing nothing into the string.

This problem is easy to solve by passing std::ws (a manipulator which discards leading whitespaces from an input stream) :

This succint version is valid too:

And here is how the ranges join the party:

In another recurring pattern the input appears in this form:

N T
a1 a2 ... aN
t1
t2
...
tT

N is the length of a sequence of numbers, T is the number of test cases. Generally this kind of challenges require you to operate on the sequence with some kind of clever pre-processing and to use each test-case value (or values) to perform an operation quickly. For example, for each test-case value t, output the sum of the first t elements in the sequence.

Here is the simplest code to use for reading inputs in this scenario:

 

We’ll meet again this problem later in this series.

More complex input patterns can be handled by combining the examples above.

 

Printing answers

 

Just a few lines on output. Obviously, by using std::cout and some convenient standard abstractions like ostream_iterator.

Often the answers are just one number or two. In this case, just send them to the standard output applying the required formatting rules (e.g. space separated):

I generally don’t use endl because it causes a stream flush and this may negatively affect the performance. For this reason I use just “\n” out of habit. In certain situations a printf is probably shorter, but it’s still error prone: not only if you write a wrong format, but also if you forget to update it in case you change a type (e.g. imagine you solve a problem by using int and then you change it to unsigned long long). With streams you don’t need to change anything because they automatically pick the correct operator<< overload.

When you need to print a sequence – like a vector – just use copy + ostream_iterator:

Note that the last separator is printed after the back element. This means extra care is needed to avoid it. For example:

Maybe in C++17 we’ll use the trailblazing ostream_joiner to save the extra line – since the “last” separator is not outputted at all:

Another worthwhile scenario is when you need to print floating point numbers and the output is expected to be an approximation of a certain number of digits. fixed and setprecision are good friends in this case. For example:

will print num1 and num2 with exactly two digits after the comma:

will print:

1.00 1.24

In case you need to print in a loop, it’s a good habit to set stream manipulators once, before the iteration:

I’ll be back on some standard mathematical utilities (e.g. rounding) later in this series.

 

Pushing streams to the limit

 

Before concluding, it’s worth sharing a couple of “beyond-the-basics” examples. Sometimes just configuring streams options is enough for solving some problems.

The first challenge required to tokenize a string based on given delimiters and to print the obtained tokens to the standard output, one per line. In other languages like Java you could use a StringTokenizer – indeed lots of Java guys use this opportunity on CC websites. Note that more complex challenges where parsing is the main topic does not allow you to adoperate standard things like streams or tokenizers (in other languages) – and they won’t be as efficient as the challenge requires!

This solution is not intended for production. So please don’t post comments like “streams are slow” or “who uses streams to tokenize?”. Here we have a different scenario. This code can be easily plugged into challenges requiring some basic parsing. By default streams skip whitespaces, here we need to skip also other delimiters.

Ranges library provides views like split and delimit for this kind of things:

Anyhow, let me go back to my modest C++14 empty sheet on HackerRank, I have only 30′ to complete the contest and Java boys are submitting two-line solutions…should I roll my own mini-parser?

C++ has a (a bit old-fashioned?) concept called facet to handle specific localization aspects. Basically, each facet corresponds to a precise localization setting that can be tuned. For example, the ctype facet encapsulates character classification features – it can answer questions like “does this char is a space?” or “does this char is a digit?”. One can inherit from a base facet and change default behavior.

For our task we can inherit from ctype<char>, a specialization of std::ctype which encapsulates character classification features for type char. Unlike general-purpose std::ctype, which uses virtual functions, this specialization uses table lookup to classify characters (which is generally faster). Here is a solution to the challenge:

This requires a bit of clarification if you don’t know facets at all: ctype<char> uses a table to associate each char to its “classification” (this table is, indeed, called classification table). The table size is standard (ctype<char>::table_size – at least 256). We just need to set char delimiters to ctype_base::space. All std::istream formatted input functions are required to use std::ctype<char> for character classing during input parsing. For example, istream::operator>> asks to ctype::is if a char is a space (aka a delimiter). Under the hood ctype<char> lookups the internal table.

Don’t worry about the lonely custom_delims allocation, the locale guarantees that the new facet will be deleted as soon as it’s not referenced anymore (facets are reference counted – another possible performance penalty, in addition to virtual function calls).

Although I never use such approach for parsing in my production code, in Competitive Programming it may be acceptable. For example, on HackerRank I submitted this code against a test-case requiring to split a string of 400’000 chars and it took (output included) 0.05s – within the time requirement of that challenge. And this code is easily reusable.

I promised two examples. The other was about number punctuation: given an integer number, print the string representation of the number, comma separated, supporting different comma styles – e.g. US 1000000 is 1,000,000, instead Indian 1000000 becomes 10,00,000. Another time, we may use the standard localization support:

The code is quite self-explanatory. For more details I encourage you to have a look at numpunct facet.

Hope you enjoyed this post because sometimes stream capabilities suffice in such competitions – consider them before rolling your own alternatives.

 

Summary

 

A brief recap of some weapons to add to our “pragmatic C++ competitive programmer toolkit”:

  • Usually I/O is not the point of the exercises, so using streams is fine
  • To read a sequence of values into a vector use:
    • vector::reserve(N), and
    • copy_n + istream_iterator + back_inserter
  • To print sequences use copy + ostream_iterator
  • To print floating points rounded to the nth-digit use fixed << setprecision(n)
  • If parsing is not the point of the challenge, consider localization features

In the last two years I have turned into a Competitive Programming aficionado, mostly active on HackerRank, less on TopCoder and CodingGame. This kind of websites enable coders not only to compete with others, but also to practice by solving several categorized exercises. After you write your program into the browser, they run and test it against some prepared test-cases. In all probability you can use your favorite language because they support pretty much everything, but I consider really precious spending time with other languages as well.

When I started I just wanted to challenge myself and push my brain beyond its limits. I really appreciated that facing with some kind of problems was a good excuse to explore new areas of CS and learn something new. Even if at that time it was not my target, competitive programming is a great way to prepare for technical job interviews.

Speaking in general, there are several reasons to get involved in Competitive Programming. I found a very nice article about that here. I quote the point that was the most important for me when I started: it’s guaranteed brain excercise: “On any given day, what you’re asked to do at work may or may not be challenging, or you may be exercising one of the many non-technical skills required of software developers. By practicing regularly with competitive programming problems, you can ensure that the coding part of your brain receives a regular workout. It’s like one of those brain training apps, but for real skills.”

Regardless of your programming language, spend some time on competitive coding.

Why C++ in Competitive Programming?

 

If you solve on websites like HackerRank a challenge then you may peek into other people solutions. Looking at C++ solutions, I have found a lot of “C++ people” using C++ mostly as C. Many people don’t consider (don’t know?) what the standard offers. On the other hand, I find it natural to face challenges with modern C++ by my side, coding as simply as possible. For example, often I ask myself: may I use an algorithm instead of this for loop? This attitude is worthwhile here as it has been for years in my daily job.

I realize the word modern is overloaded: in the last years we all have probably heard the expression modern C++ everywhere and sometimes it looked like a buzzword. I mean using the full standard toolset of C++, without external libraries, nor over complicated abstractions. Mainly because in competitive programming you don’t have libraries (sometimes you don’t have the standard library neither) and you cannot lose time with too much generalization. Many times the same results fit real life development.

It’s worth noting that I’m not going to explain things like “how to make a linked list”. Excellent algorithms & ds courses/books/series have been developed for this purpose. Rather, expect to see how I used a forward list to solve a problem, where I found necessary a multiset, or when lower_bound saved my life.

I have told to some friends about this series and they suspected that C++ was not the right language for talking about Competitive Programming. Actually I had a look at the top ranked people on websites like TopCoder and HackerRank and C++ was always there (together with Java and Python, mainly). I found another indicative example on Google Code Jam: more of the higher-performing contestants code in C++.

I’m not surprised at all.

Certainly C and C++ are considered the lingua franca of algorithms and data-structures, but I think the main reason is the control C++ offers and its independence of the paradigm:  it does not force you a single “way of doing” to solve a challenge.

My friend Alessandro Vergani has another sensible idea: traditionally C and C++ have a poor standard library compared to newer languages like Java and C#, for this reason C/C++ programmers are at ease with writing everything they need from scratch – and often too much. This means they are fluent with coding things that in other languages are took for granted.

Moreover, I think since lots of participants are students, C++ is very common as a “first language”.

 

Challenges are about trade-off

 

My purpose is solving challenges without reinventing the wheel. This means if I can use standard things to make all the test cases pass, I’ll do it. Obviously, I’ll justify my decisions and I’ll discuss a bit. For example, what about this snippet for pretty printing a non-zero unsigned 32-bit integer N into its binary representation:

It’s simple, but it’s not as efficient as it can be if written in other ways.

Does it suffice to solve the problem?

Maybe.

If so, I would be happy with this simple solution. If not, the challenge is asking me to find better ways to solve the problem.

Challenges are about trade-off. The requirements and the constraints of the problem matter a lot. They are a rough approximation of a real problem. Often you can exploit the constraints of the problem and take a shortcut. This may signify optimizations, or simplification, etc. Very often you have to exploit the constraints of the problem to solve it.

Many times these shortcuts are not so far from real life needs.

 

Standard reasoning

 

How many times you heard commands like “use standard algorithms” or “use a container”. With luck, we all know the classical resons for doing that: efficiency, correctness, maintainability. Now there is another fact we could care about: the future of our language.

Let me articulate.

Probably the biggest limitation of the STL is the lack of composability. One of the most mind-changing addition to C++17 will be Eric Niebler’s ranges. Ranges are basically a shift from iterators to a superior abstraction. Ranges enable programmers to write fluent and – hopefully – efficient algorithms.

Coding with ranges produces “pipelined” transformations. For example, to generate an infinite list of integers starting at 1, consider only even values, square them, take the first 10, and sum them:

It’s not my intention to talk about ranges and I am sure you rule them better than me (if not, give it a try and refer to the documentation). My point is: Eric’s proposal includes standard combinators like transform and take, and mutating ones like sort and unique. They are essentially the counterparts of those in the STL. Thinking in terms of standard blocks help embrace ranges effectively. If you are used to employ standard functions, you are already confident with several range views and algorithms.

When I can, I’ll try to show my solutions by using ranges as well. Clearly ranges are not part of the standard yet then these solutions are not intended to run on competitive programming sites yet. I’ll adoperate Clang on Visual Studio 2015 for my attempts.

 

Standard flexibility

 

Many people don’t employ standard algorithms massively. For some of them the reason is that algorithms are often closer to math than to programming – even if programming is math…

Consider the simple task of finding the minimum difference in a sequence of ascending sorted numbers. For example, in this sequence:

[10, 20, 30, 100, 200, 300, 1000]

10 is the minimum absolute difference (e.g. 20-10).

This is an explicit solution:

That solves the problem in a single pass.

The same problem written in Python:

minDiff = min([r - l for l, r in zip(elems, elems[1:])])

That is completely different because it employes a functional block that C++ misses: zip. Basically it combines elements pair-wise from N sequences (two in this case).

Look more closely at the C++ solution.

It’s basically an accumulation, right? In each iteration we calculate the difference between two adjacent elements and then we update the global minimum. But it’s not a std::accumulate because that function cannot operate on two sequences at the same time – at least without special iterators.

Any idea?

I am sure you found it. Your standard friend is inner_product:

We’ll turn back to inner_product later in this series because it’s powerful, but just to recap: the simplest overload of inner_product returns the result of accumulating a starting value with the inner products of the pairs formed by the elements of two ranges. That is:

init += range1[0]*range2[0] + range1[1]*range2[1] + ... + range1[N-1]*range2[N-1]

There is more: both sum (+) and multiplication (*) can be customized. In my example, I replaced accumulation with min and product with absolute difference. I also set the initial value to the maximum possible int value – in math I would set it to infinity, the identity of the minimum operation (e.g. min(infinity, value) is value). Note also that by passing [begin, end-1] and [begin+1, end] I basically obtained these “pairs”:

range[0], range[1]
range[1], range[2]
...
range[N-2], range[N-1]

inner_product has a name closer to math than to CS, but it is actually a general function, that hides a rich flexibility. When I say flexibility I am not saying composability. Flexibility means that the same function can be adapted to different scenarios. Standard functions generally have a great flexibility.

But flexibility can lead to misuses…

Suppose after we calculate the minimum difference we want to output the pairs of elements with that difference. Recalling the first example:

[10, 20, 30, 100, 200, 300, 1000]

The minimum difference is 10 and the pairs of elements differing by this value are:

10, 20
20, 30

So basically we need to iterate over adjacent elements and print them if their difference is 10:

[20-10, 30-20, 100-30, ...]

Is there any standard algorithm for this purpose?

std::transform is really close, but it does not really perform what we need mainly for two reasons: first of all, it does not guarantee in-order application of the custom function – or in-order iteration – and second, transformation is N to N, that is each element in the source gets mapped into another thing.

Ideally we need something like a zip and a copy_if or copy_if and a special iterator type wrapping two adjacent elements at the same time – things that are, for example, in boost. For this reason, I think a for loop is the best choice in “bare” C++:

Looking at this code, some C++ hackers may force std::equal to do the same thing, getting some sugar:

The sugar is given by not accessing the vector and not writing the for loop explicitly.

The problem is that the snippet is unclear and hackish: std::equal has not been designed to do such things. What a programmer expects is just a pair-wise comparison, eventually using a custom predicate. You know, don’t fall into the common “if all you have is a hammer everything looks like a nail”. Just say no. The more you practice with standard algorithms, the more they will pop up in case they match. Pretty much automatically.

Flexibility comes with responsibility. Standard algorithms constitute well-known patterns. They help understand the code because other people have clear expectations from them. Hacking standard algorithms can lead to disasters.

If you miss a block, at least two ways are doable: if you don’t need to reuse it, go with a local solution. If reuse is important (and generally it is) then write a function designed to be reused. This requires some care, and keeping in mind how standard C++ algorithms are designed can be a good starting point.

Since “some care” requires time, in competitive programming this is not always feasible. For this reason people tend to go with custom solutions, even for summing the elements of an array. In this series I have more time and there is nothing to prevent me to articulate and to show some solutions based on simple functions that C++ misses. Pragmatic competitive programmers generally keep such functions and snippets on hand. Sometimes the standard already provides such functions, and it’s just the matter of practicing.

Although standard algorithms are really basic, they support a rich flexibility, as I said. This means you can roll your own special iterator and solve the problem in a single pass – imagine also ostream supports operator<< for pairs:

Is it worth? There is not a single answer to this question. It’s really contextual, it depends on where you work at, if your codebase is too much complicated to need these abstractions, etc. Whenever you introduce (or use from a library) a new block (e.g. adj_it), it should be learnt by you and – hopefully – who works on your codebase.

Ranges should help a lot in these scenarios because in addition to a rich flexibility they have a strong composability. Let me turn the “partial C++17 support” on and show you my range-based solution to the initial problem:

It seems completely different from the inner_product one, but what it really changes is the way of thinking the algorithm. This solution is probably closer to Python than to C++, isn’t it? That’s because more blocks get available and composition plays an important role. This makes our way of thinking the algorithm closer to other paradigms. Again, this is an act of responsibility: choosing between – say – a for loop and a range-based solution can be disputable.

Are the inner_product and the ranges based solutions hard to understand? Definitely they are for programmers who don’t know standard idioms. It’s the same for map.insert that does not insert at all if the element is already there. Is std::map misleading? I know people who spend more time at dumping on standard things than the time they would need to learn standard things properly. Mastering language patterns is the key, and in the next posts I’ll show some patterns involving standard algorithms and containers.

In this series, first I’ll consider pre-C++17 solutions and whenever I can I’ll outline possible C++17 answers.

An invitation about ranges: please, post comments with your suggestions/corrections/opinions on my solutions because I think use cases are important both for Eric and for the whole C++ ecosystem.

 

What’s next

 

I don’t have a precise idea for the other posts yet. It’s possible I’ll discuss the usage of a certain standard algorithm in different scenarios, or how to deal with some kind of problems, or how to use a container to solve certain challenges, etc.

Definitely, since this series is about competitive programming, I’ll spend some posts on some peculiar aspects of challenges like paying attention to the constraints of the problem, or choosing the proper data type for the solution.

The next post will be kind of preliminary to the others. It’s basically about input and output.

Final note about code snippets

You probably noticed I used gist to render code snippets in this post. For the first time I needed to completely bypass WP: yesterday this post was broken because of snippets, with no apparent reason. I tried to fix them but after one hour of attempts I gave up. WP is the enemy of programming bloggers. I know, I should move to Jekyll or something better, but WP still has some advantages. For this reason I’m experimenting this “embedding Gist feature”. The procedure is a bit longer (create gist, paste code, give a sensible name, save, copy the URL into the post) but I think I would spend the same time at fixing WP troubles, so it’s ok for now.

A couple of weeks ago I found a simple bug in the dusty corners of a production repository I usually work on. A certain feature was rarely used and it seemed to be covered by a test…Actually it was not and some years ago a guy, feeling guarded by this test, changed the code bugging the feature, but nobody complained. Recently a user tried to resume this old feature but it didn’t work as expected.

I don’t want to bother you with all the details of the story but just to give you a bit of context: there was an old piece of code reading a small file through C FILE handles. Some years ago that piece of code was migrated to C++ streams (since files to read were really small and simple) and a silly bug was introduced. Since this bug was really simple I wondered if it was caused by inattention or ignorance, then I had a chat with the programmer who committed the change and I discovered it was caused by the latter reason. The fix was really easy.

Some days later I discussed about this problem with some friends and I realized they were unaware of this problem too. So I decided to write a short post about this story, maybe it is useful to other coders.

Imagine you are using streams to read some data from the standard input. This is what the input looks like:

number
some words in a line
number
some words in a line
...

And then imagine the following code reading that input:

int num; string line;
while ( (cin >> num) && getline(cin, line) )
; // something

Did you spot any problems?

If not, don’t worry, it’s a bit subtle.

Consider the invisible characters contained in the input stream:

10'\n'
some words'\n'
...'\n'

Actually this is not formally true on Windows, but in general you have a LF char at the end of each line. Let’s follow the code flow:

  • cin >> num correctly reads the int, stopping at (in the language of streams: “detecting but not consuming”) ‘\n’
  • getline(cin, line) now reads the next line until it encounters a line separator (‘\n’ by default). But ‘\n’ is still in the stream buffer and then getline returns immediately, storing nothing in line.
  • Again cin >> num is evaluated but this time it fails, because the stream is not fed with an int. failbit is set then. The loop terminates.
  • The user complains because the feature does not work as he expects. Ok, sorry this is not part of the code flow…

We just experienced a difference between operator>> and getline: the first skips any leading whitespace (actually any separator – according to the locale in use) before performing the read operation, instead, the second does not.

Basically, it has to do with the difference between formatted and unformatted input function. Stream operators (like operator>> for strings) belong to the former category, getline to the latter. In short – among other differences – formatted input functions skip leading separators (e.g. whitespaces, LF) by default, unformatted functions do not.

The long story is: both formatted and unformatted functions create basic_istream<CharT>::sentry objects for preparing input streams for I/O (e.g. checking the validity of the stream). One of the operations performed in the sentry constructor is skipping leading whitespaces. For deciding whether skipping or not it uses two information:

  • a bool parameter  passed to the constructor, that is false by default. Don’t get confused: it’s false when you want the sentry object to skip whitespaces (in fact, it’s generally called noskipws – e.g. _Noskip on Visual Studio).
  • ios_base::skipws flag (set or not on the stream object).

If _Noskip is false and the ios_base::skipws is true then leading whitespaces will be skipped.

I am sure you already imagine the rest of the story: when a formatted function creates a sentry, the parameter is left to its default value (false) and since cin‘s ios_base::skipws is true, operations like cin >> i work as expected even if some whitespaces stand in front of the int value. Conversely, unformatted functions create sentry instances by explicitly passing true to the constructor. For this reason the lonely leading ‘\n’ is not discarded.

[note]

Beware something about formatted/unformatted functions is changed between C++98 and C++11, in particular about istream& operator>>(streambuf*). In fact in C++98 it was a formatted operation, now it is unformatted.

[/note]

Why does getline preserve leading separators? I think because it’s kind of raw read operation. Note that if the delimiter is found, it is extracted and discarded (e.g. it is not stored and the next input operation will begin after it). This is important, because it enables such a code to work as expected:

stringstream ss("the first line\nthe second line)"
while (getline(ss, line)) 
{ ... // line does not contain '\n' at the end

How we fixed this issue?

The simplest thing you can do is just:

while ( (cin >> num >> std::ws) && getline(cin, line) )
;

The left hand side reads the int and then skips leading separators from the stream. std::ws is an input manipulator created for this purpose.

A bunch of other solutions are possible. For example the one using ignore:

while ( (cin >> num).ignore(numeric_limits<streamsize>::max(), '\n') && std::getline(cin, line))

Here we discard as many leading separators as possible, that is until either count characters are discarded, the delimiter (specified by the second parameter) is found or the end of the stream is reached.

Not only is the former solution simple and effective enough, but it also prevents oversights like:

10'\n'
'\n'
some words

Above the user left an empty line after the first number. The std::ws solution does not break, the ignore one does instead. On the other hand, std::ws solution does not preserve leading whitespaces, if you need them. But it was not our case (you can imagine the final code looked a bit more defensive than the one I showed here, but the main observations hold).

One can also develop a proxy object to allow code like this:

cin >> num >> std::ws >> as_line >> line;

as_line may also embody the std::ws part:

cin >> num >> as_line >> line;

It’s not hard to code such a machinery. For example:

struct lines_proxy
{
	istream& operator()(string& s)
	{
		return getline(is >> std::ws, s);
	}

	istream& is;
};

struct line_t {} as_line;

lines_proxy operator>>(istream& is, line_t)
{
	return{ is };
}

istream& operator>>(lines_proxy p, string& s)
{
	return p(s);
}

...

while (cin >> num >> as_line >> line)

Just for fun.

It’s over for today. The leading actor of our story was a really silly problem but the lesson learned was interesting: even if streams were designed as a “formatted” abstraction on top of I/O buffers, unformatted operations are still there. Mixing formatted and unformatted operations should be done with care.

This post is just a reminder to myself because I fell for it, again…Well, let me step back and explain the scenario.

Suppose we are using this generic approach for memoization:

template <typename R, typename... Args>
auto memoize(R(*fn)(Args...))
{
    std::map<std::tuple<Args...>, R> table;
    return [fn, table](Args... args) mutable -> R {
        auto argt = std::make_tuple(args...);
        auto memoized = table.find(argt);
        if(memoized == table.end())
        {
            auto result = fn(args...);
            table[argt] = result;
            return result;
        }
        else
        {
            return memoized->second;
        }
    };
}

Suppose also – at some point – two new requirements pop up:

  • ability to remove/update some entries of the table – because something changes somewhere else and those results become old,
  • ability to switch to other functions (the table is not needed there).

We decide to store the table and the lambda into a struct. For the sake of simplification, this is a specialized version of such a class using a toy-function simulate:

struct Memoization
{
    Memoization()
    {
        calc = [this](int i)
        {
            auto memoized = table.find(i);
            if(memoized == table.end()) {
                auto result = simulate(i); // <- somewhere
                table[i] = result;
                return result;
            } else {
                return memoized->second;
            }
        };
    }

    function<int(int)> calc;
    std::map<int, int> table;
};

Even if this design is probably silly, we can easily satisfy both the requirements (e.g. calc can be set to something else, and table is fully accessible).

This code is fine for rapid prototyping, so we don’t refactor it yet but instead we experiment a bit more. For example, we create a factory for creating different Memoization instances depending on some configuration. Each configuration merely results in a new core function to use (e.g. remember the second requirement). Since it’s prototyping, we leave the constructor as is and we instead set the function by hand:

Memoization CreateMemo(const Configuration& config)
{
   Memoization memo;
   // using config to create memo
   // e.g. memo.calc = ...
   return memo;
}

We try this code and we note that it behaves differently on two compilers: on Visual Studio 2013 we get a crash at some point while calling calc(), instead on clang it seems to work smoothly. We start debugging and we immediately spot what’s going on…Do you note that we are one step away from falling into a dangling reference problem? Actually, this accident happens on both clang and Visual Studio, but some (un)lucky condition makes this work on the former.

The culprit – in this case – is RVO, but the issue is…both capturing this into calc – that is a member variable – and copying/moving *this.

By capturing this, we have coupled calc to this->table, that won’t change anymore – say when you do move (or copy). this has not special treatment inside the callable object created by the lambda expression. It’s just a Memoization pointer and when the lambda gets moved (or copied), so does the pointer. Shallowly.

In our factory function, RVO is probably working a bit differently between clang and VS. VS is not using RVO here, clang instead uses RVO and the problem is apparently concealed. By disabling RVO on clang (e.g. -fno-elide-constructors), we get the same problem found on VS.

Did you get the point? In case of the original “memoized” version of calc (which captures this), after the move (or the copy), the returned Memoization instance has a reference to the local memo->table, not to the new one. Finally, the local temporary instance is destroyed and we get a dangling reference. This problem is subtle since the code could still “work” under certain conditions – e.g. RVO. It should be clear that copying instead of moving has the same effect.

Maintaining the original design, the problem can be quickly solved, for example by constructing the object directly:

class Memoization
{
public:
   Memoization()
   {
    // same as before
   }

   Memoization(function<int(int)> calcFn)
      : calc(calcFn)
   {
   // ...
   }
// ...
};

Memoization CreateMemo(const Configuration& config)
{
   if (config. ...)
      return {...} // won't copy nor move
   //...
}

But the main issue holds and this could lead to disaster:

Memoization m1;
auto m2 = m1; // [this] in m2.calc points to m1

Not only is it dangerous, but also wrong: you probably expect each Memoization instance has its own copy of the table – i.e. for this reason a solution – say – with shared ownership of the table doesn’t fit well.

At the end of the story we changed our design and we came up with another better solution. But this is not the main point of the article. Even if my example is probably goofy, this experience left a valuable lesson: capturing this into a member variable lambda is valid C++ code and may cause headache if we copy/move *this. Sometimes I think we have a duty to set some limits, for preventing traps other people could fall into. For this reason, I came to a couple of observations:

(You understand that moving instead of copying does not twist the meaning, so let me just mention copying so I do not need to write copy/move every time).

First, we usually don’t need to capture this into a member variable lambda at all. Exploring better ways is always more advisable.

Some of you could still complain that C++ is missing an opportunity by letting this behavior go undisturbed. You could expect the compiler magically sets the this pointer to the copied instance, don’t you? Honestly, I have no strong opinions on that, I’m just thinking aloud.

Second: we can judge pragmatically. Capturing this into a member variable lambda and copying *this just do not get along. Doing either is realistically better than adding some special treatment.

For this reason, I see a terse idiom: either capture this into a member variable lambda or copy *this (or neither).

This is eventually another subtle case the Rule of Zero does not cover and – in case you cannot live without capturing this – I think deleting copy and move operations in the host class is such a desirable design decision to apply (and to document?) – being understood that maybe you don’t need to capture this into a member variable lambda at all.

Last month Marco Foco and I facilitated a workshop on refactoring legacy C++ code. It was an improved version of the same workshop we presented at the Italian Agile Day in November, with Gianluca Padovani as well.

To give you a bit of context, some days before the attendees cloned a certain Git repository we indicated and they compiled the code (by using CMake to generate the projects in their environment) on their machines. The workshop was divided in 4 parts, each one focusing on a C++ theme. They were: productivity, memory management, algorithms and generic programming. During each part, we first spent 10 minutes explaining a few C++11/14 concepts and then we gave 25 minutes to work on some refactoring exercises. At the end of each part, a brief retrospective.

In this post I’m going to describe three pitfalls attendees fell into, related to initializer_list  at least at first sight.

The workshop code was a simple version of Yahtzee, the famous dice game. It was test-driven and, among others, we wrote a test suite covering the score calculation. For instance, suppose a player rolls the dice getting:

2 2 3 3 3

She gets a full house, or 25 points. A class is responsible for recognizing and calculating this kind of stuff. To roll the dice, another class is involved, a sort of “IDiceRoller”. It is an interface that provides only one function:


virtual void Roll(int(&dice)[5]) = 0;

In the test suite, we implemented a simple fake object (not a mock – that could be an exercise) to manipulate and control the dice. Imagine:


class FakeDiceRoller : public IDiceRoller
{
 public:
  FakeDiceRoller() { ... set _dice[i] = 1 ... } 
 
  void AssignDiceValues(int values[5]) { ... copy values to _dice ... }

  void Roll(int(&dice)[5]) { ... copy _dice to dice ... }
private:
  int _dice[5];
};

// in a test fixture

// FakeDiceRoller _roller is a member variable

int dice[5] = {1,1,2,3,4};
 _roller.AssignDiceValues(dice);

FakeDiceRoller had intentionally a poor interface and design. The point was: how could you improve it? Suppose you couldn’t change the interface of the domain interface IDiceRoller, as – likely – in the real life. The first series of exercises was about productivity. Sure, AssignDiceValues was one of the point we wanted participants to think about, At some point they gave a try:

_roller.AssignDiceValue({1,2,3,4,5});

They compiled and…they failed.

“Cannot convert initializer list argument to ‘int*'”.

People started trying to figure out why initializer_list was not covertible to int[]…

“This has nothing to do with initializer_list”. I stated. “This is the language and it’s saying the function parameter int values[5] is just int* dice, you cannot initialize int* from {1,2,3,4,5}”.

Then a gentleman took the floor and shouted “use the same signature as Roll, accepting a const reference to array instead of a non-const reference”. That was:

void AssignDiceValues(const int(&values)[5])
// usage
_roller.AssignDiceValue({1,2,3,4,5});

It was fine.

But we were more subtle. Now the attendees started merging two lines in one, getting code like the following:

_roller.AssignDiceValue({1,1,1,1});

Do you spot any problem here?

Now FakeDiceRoller‘s _dice is:

[1, 1, 1, 1, 0]

This is because the language zero initializes missing ints.

It happended the test at issue checked a poker of ones. And ok, we had four ones. It happended also the test was a bit wrong, because it didn’t check the 5th value of the dice (say it was a 3, the test had to check we scored a poker AND a 3). Who wrote the test made a mistake. C’est la vie.

Can our test environment prevent us doing this kind of imprudence? Now this simple requirement can be translated to a C++ exercise: we want to inline-initialize an array of strictly N elements. In short, this should fail under our conditions:

void AssignDiceValues(const int(&values)[5]);
_roller.AssignDiceValues({1,2,3,4}); // missing last die

How can this be done? I give you 3 attempts. Other solutions are possible, here I want the simplest approaches possible. The good news was that many people at the workshop suggested the last, that I think is the best.

Attempt #1: initializer_list

Since initializer_list contents is sculpted in the code – at compile-time – its .size() function should be constexpr, shouldn’t it? Yes, but from C++14:

void AssignDiceValue(initializer_list<int> values)
{
   static_assert(values.size() == 5, "Please provide exactly 5 values"); // only from C++14
   copy(values.begin(), values.end(), _dice);
}

Actually this doesn’t work yet, as a reader commented.

Attempt #2: strict_array

template<typename T, size_t N>
struct strict_array : array<T, N>
{
   template<typename... V>
   strict_array(V... vals) // no &&/forward to simplify
      : array<T, N>( {vals...} )
  {
     static_assert(sizeof...(vals) == N, "Please provide exactly 5 values");
   }
};

void AssignDiceValues(const strict_array<int, 5>& values);
_roller.AssignDiceValues({1,2,3,4,5}); // ok
_roller.AssignDiceValues({1,2,3,4}); // static_assert fires
_roller.AssignDiceValues({1,2,3,4,5,6}); // static_assert fires and array<int, 5> constructor complains

Attempt #3: just the language

We don’t want to add complexity to my framework. We don’t need static_assert nor new bizarre array types. We can use the language and bring my requirement out by design. Just needing to add a tiny level of abstraction:

class Die
{
   int value;
public:
  Die(int val) : value(val) {} // mandatory
  // operator int() { return value; } // if really needed
}

void AssignDiceValues(const array<Die, 5>& values);
_roller.AssignDiceValues({1,2,3,4,5}); // ok
_roller.AssignDiceValues({1,2,3,4}); // ko
_roller.AssignDiceValues({1,2,3,4,5,6}); // ko

This is the solution I like the most. It’s a design decision.

Two main points about initializer_list to remember when you refactor legacy “initialization” code:

  • int arr[] is int*. Don’t expect the language to magically deduce an initializer_list
  • initializer_list‘s size() is constexpr only from C++14.

Next. Another task where initialization was involved regarded the game configuration: a game could be configured with a few options. Since the codebase was an hybrid of old C++ and modern C++, a tuple was employed.

YathzeeGame game ( make_tuple(5, 6, 2) ); // 5 dice, [1..6] values, 2 players

A gentleman spotted the following in the dark corners of the codebase:

vector<YahtzeeGame> games;
games.push_back(make_tuple(5, 6, 2));
games.push_back(make_tuple(5, 6, 3));
games.push_back(make_tuple(5, 6, 4));
// other stuff

Excited about C++11, he tried to refactor:

vector<YahtzeeGame> games = { {5, 6, 2}, {5, 6, 3}, {5, 6, 4} };

And does it compile?

Yes!

No.

I’m kidding you!

I rephrase: do the following statements compile?


YahtzeeGame game { 5, 6, 2 }; or YahtzeeGame game { {5, 6, 2} };

No, they don’t neither. Some participants asked “why is not initializer_list supported here?”.

“initializer_list is not guilty”, I replied. First: how can one expect an initializer_list to be used to contstruct a tuple? initializer_list is – by-definition – homogeneous! Just the opposite of tuple. tuple should have a constructor taking…initializer_list<?>. Some people started likening tuple to pair: “I can do it with pair”.

Yes, you can do with pair because the real reason is tuple’s constructor, that is explicit, and – as you know – copy initialization considers only non-explicit constructors. That is, you can do:


tuple<int, string, foo> t {10, "hello", {fooArg1, fooArg2}};

But not:


tuple<int, string, foo> t = {10, "hello", {fooArg1, fooArg2}};

Nor:


tuple<int, string, foo> make_my_tuple() {

   return {10, "hello", {fooArg1, fooArg2}};

}

So you may refactor the initial code by adding make_tuple:

vector<YahtzeeGame> games = { make_tuple(5, 6, 2), make_tuple(5, 6, 3), make_tuple(5, 6, 4) };

Also here initializer_list was in the clear. When something is wrong with initialization, since curly braces initialization (aka uniform initialization) and initializer_list share the same syntax, and since almost all the standard containers support initializer_list construction, someone could jab at this type. As a reader commented, N4387 proposes (among other stuff) getting rid of this limitation.

The third and last example is another story.

To calculate the scores, a class with a CalculateScores function was provided. This function was monolithic, imagine a big if cascade:

if (...single dice...) {
...
}
if (...pair dice...) {
...
}
if (...tris dice...) {
...
}
if (...poker...) {
...
}
etc.

We proposed to decouple this function and make it modular. This way one can create several versions of the game, for example one with no special points (e.g. no full, poker, straight), another with extra points, etc. People designed a simple IRule interface, providing a function:

virtual void Apply(const GameState& state, ScoreTable& scores) = 0;

ScoreTable was already in the code and it just stored the results of the calculation. The idea was to apply rules in chain. Straightforward.

A funny anecdote: at some point I asked “how can you improve this if cascade?”. One person replied “we can use a switch-case”. I responded: “yes but…it’s pretty much the same. What can we do from a design point of view to make this code more modular?” Another guy said “we can design an interface and several concrete classes”. And suddenly the person who proposed the switch-case got up and left the room! Ouch…is an interface so bad?!

No more chatting! People coded this interface, created the rules and…they had to store them somewhere. They opted for a vector of unique_ptrs:

vector<unique_ptr<IRule>> rules;

And they serenely wrote:

vector<unique_ptr<IRule>> rules = { make_unique<Single>(), make_unique<Double>(), make_unique<Tris>(), make_unique<Full>(), ... };

And they got impatient for testing their code, having unit tests from their side – contrary to what they have at work:)

I felt a tremor in the force…

“Noooo. Another compiler error”😦

Said desperate programmers whining from the trenches.

“call to deleted constructor of ‘std::unique_ptr<Single, std::default_delete<Single> >”

“What the fuck?” Some of them kindly complained!

This time, they really made initializer_list fell guilty. And this time they were right. initializer_list doesn’t support move-only types. The main reason is that its begin() and end() return const pointers. There is a proposal to address this issue and several smart guys advanced their idioms – for example here.

As before, I wanted a simple solution for my modern C++ novices, to let them play and experience with C++11/14. I seized the moment: “guys, let’s do a simple exercise with variadics “:

auto rules = CreateVector<unique_ptr<IRule>>( make_unique<Single>, make_unique<Double>(), ... );

The idea was very simple and so was the implementation:

template<typename T, typename H>
void CreateVectorImpl(vector<T>& v, H&& single)
{
   v.emplace_back(forward<H>(single));
}

template<typename T, typename H, typename... Tail>
void CreateVectorImpl(vector<T>& v, H&& head, Tail&&... tail)
{
   v.emplace_back(forward<H>(head));
   CreateVectorImpl(v, forward<Tail>(tail)...);
}

template<typename T, typename... Tail>
vector<T> CreateVector(Tail&&... tail)
{
   vector<T> v;
   CreateVectorImpl(v, forward<Tail>(tail)...);
   return v;
}

Alessandro Vergani (who were there to help us) sent me this (specific – but slick) solution:

template <typename Type>
void setup_rules(vector<unique_ptr<IRule>>& v)
{
   v.emplace_back(make_unique<Type>());
}

template <typename Type, typename Type2, typename... OtherTypes>
void setup_rules(vector<unique_ptr<IRule>>& v)
{
   v.emplace_back(make_unique<Type>());
   setup_rules<Type2, OtherTypes...>(v);
}

template<typename... Types>
vector<unique_ptr<IRule>> CreateRules()
{
   vector<unique_ptr<IRule>> rules;
   setup_rules<Types...>(rules);
   return rules;
}

// usage: auto rules = CreateRules<Single, Double, Poker>();

Wrapping up the story, I believe people at the workshop tried to burden initializer_list too much. They got errors on something related to initialization with curly braces and they accused initializer_list. In the first case, the main misunderstanding was related to the language itself: int[] is just int*, in C++11 as in C++98. Rectifying was quite simple, by using a const reference to an array. initializer_list doesn’t have to do with that, not even here. It’s the language. And just by using the language we addressed the other requirement about prohibiting “uninitialized” dice. Here, some people thought they could just static_assert initializer_list’s size. I deem it’s not worth.

At first sight, the second case is even more related to initializer_list, because every container is constructible from initializer_list. Why tuple differs? If people don’t think about the mathematical difference between initializer_list (aka: homogeneous) and tuple (aka: heterogeneous) they can fall into a trap. Pair is the same story, but curly braces are because of uniform initialization. And pair’s  constructor is not explicit, thus copy initialzation is possible and the trap is just veiled.

In the last example, initializer_list tried to escape through the window, but this time it couldn’t. Imagine initializer_list as arrays, globally stored somewhere. Even if they are (maybe) used only once (in the line you perform the initialization), the compiler is solely responsible for their state. We know there are many workarounds but I’d really like having an official feature in the standard to address this issue (e.g. N4166).

[Note: thanks to Davide Di Gennaro for having reviewed this post and for having suggested some improvements. A paragraph has been completely written by Davide.]

[From Wikipedia] In programming, named parameters refer to a computer language’s support for function calls that clearly state the name of each parameter within the function call itself. A function call using named parameters differs from a regular function call in that the values are passed by associating each one with a parameter name, instead of providing an ordered list of values.

Compare with a traditional positional function call:

createArray(10, 20); // what does this mean, precisely?
createArray(length=10, capacity=20); // oh, I see...
createArray(capacity=20, length=10); // same as previous

A typical example:

// some pseudo-language
window = new Window {
   xPosition = 10,
   yPosition = 20,
   width = 100,
   height = 50
};

This approach is especially useful if a function takes a large number of optional parameters, and users are usually going to accept the default for most of them.
Several languages support named parameters (e.g. C#, Objective-C, …). C++ does not. In this post, I’m going to explore some of the classical ways to emulate named parameters in C++ as well as mention new approaches.

Comments

Only to mention, the most trivial way to emulate named parameters is through comments:)

Window window {
 10, // xPosition
 20, // yPosition
 100, // width
 50 // height
};

This approach is very popular among windows developers, as MSDN pubishes Windows API usage examples with such comments.

Named Parameter Idiom

Imported from Java programming style is the Named parameter idiom (and this is just another post about). The idea is simple: create a proxy class which houses all the parameters. Optional ones are settable via a fluent interface (e.g. by using method chaining):

// 1
File f { OpenFile{"path"} // this is mandatory
   .readonly()
   .createIfNotExist()
   . ... };

// 2 classical version (not compliant with the auto-everything syntax)
File f = OpenFile { ... }
   .readonly()
   .createIfNotExist()
   ... ;

// 3 for auto-everything syntax just add a layer (I prefer 1)
auto f = CreateFile ( OpenFile("path")
 .readonly()
 .createIfNotExists()
 . ... ));

OpenFile is a sort of parameter object and File’s constructor accepts an OpenFile instance. Some authors (for instance, here) argue that OpenFile should have only private members and declare File as friend. This makes sense if you want to add more complex logic to set your parameters via the fluent interface. If you merely want to set plain parameters then a public struct suffices.

In this approach:

  • mandatory parameters are still positional (as the constructor of OpenFile is a regular function call),
  • optional parameters must be copy/move-assignable (basically, settable) – possible runtime penalty,
  • you need to write an extra class (the proxy).

I – almost – never found usages in critical code.

A final note: the fluent interface can be polymorphic as I wrote more than two years ago.

Parameter Pack Idiom

Similar to the one above it’s the parameter pack idiom – from Davide Di Gennaro’s Advanced C++ Metaprogramming – a technique using proxy objects to set parameters via assignment operator (=), resulting in a sweet syntactic sugar:

MyFunction(begin(v), end(v), where[logger=clog][comparator=greater<int>()]);

The actors involved here are:

  1. logger and comparator are global constants; the assignment operator just returns a wrapped copy of the assigned value,
  2. where is a global constant of type “parameter pack”, whose operator[] just returns a new proxy that replaces one of its members with the new argument.

In symbols:

where = {a, b, c }
where[logger = x] → { a,b,c }[ argument<0>(x) ]  →   {x,b,c}

Sketching an implementation, just to give you an idea:

// argument
template <size_t CODE, typename T = void>
struct argument
{
   T arg;
   argument(const T& that)
      : arg(that)
   {
   }
};

// void argument - just to use operator=
template <size_t CODE>
struct argument<CODE, void>
{
   argument(int = 0)
   {
   }
   template <typename T>
   argument<CODE, T> operator=(const T& that) const
   {
     return that;
   }
   argument<CODE, std::ostream&> operator=(std::ostream& that) const
   {
      return that;
   }
};

// argument pack (storing the values)
template <typename T1, typename T2, typename T3>
struct argument_pack
{
   T1 first;
   T2 second;
   T3 third;
   argument_pack(int = 0)
   {
   }
   argument_pack(T1 a1, T2 a2, T3 a3)
     : first(a1), second(a2), third(a3)
   {
   }
   template <typename T>
   argument_pack<T, T2, T3> operator[](const argument<0, T>& x) const
   {
      return argument_pack<T, T2, T3>(x.arg, second, third);
   }
   template <typename T>
   argument_pack<T1, T, T3> operator[](const argument<1, T>& x) const
   {
      return argument_pack<T1, T, T3>(first, x.arg, third);
   }
   template <typename T>
   argument_pack<T1, T2, T> operator[](const argument<2, T>& x) const
   {
      return argument_pack<T1, T2, T>(first, second, x.arg);
   }
};

enum { LESS, LOGGER };
const argument<LESS> comparator = 0;
const argument<LOGGER> logger = 0;
typedef argument_pack<basic_comparator, less<int>, std::ostream> pack_t;
static const pack_t where(basic_comparator(), less<int>(), std::cout);

For the complete code, please refer to Davide’s book.

While this technique may look interesting, in practice it’s hard to generalize. In the book, in fact, it’s included as an example of “chaining” multiple calls to operator[].

Tagging

Andrzej Krzemieński published an interesting post Intuitive interface, where he mentions an alternative approach.
Named parameters are introduced by companion tags (empty structs used just to select different overloads of the same function). Notable examples of tags are from the STL:

std::function<void()> f{std::allocator_arg, a}; // treats a as an allocator instead of a callble object
std::unique_lock<std::mutex> l{m, std::defer_lock}; // don't lock now

Andrzej proposes tags to improve readability:

// not real STL
std::vector<int> v1(std::with_size, 10, std::with_value, 6);

I like his approach, but – as it stands – you possibly need to create lots of overloads and you cannot choose the order of the parameters. However, there are no requirements on copy-assignment, default values and forwarding is also clear. From the article: “However, tags are not an ideal solution, because they pollute the namespace scope, while being only useful inside function (constructor) call.”

Additionally (from the comments to the article) a reader proposes a slightly different idea that uses a proxy:

std::vector<int> v1(std::with_size(10), std::with_value(6));

Boost

Boost has the Parameter Library.

It’s possibly the most complete option if you really need named parameters in C++. An example:


// class code
#include <boost/parameter/name.hpp>
#include <boost/parameter/preprocessor.hpp>
#include <string>

BOOST_PARAMETER_NAME(foo)
BOOST_PARAMETER_NAME(bar)
BOOST_PARAMETER_NAME(baz)
BOOST_PARAMETER_NAME(bonk)

BOOST_PARAMETER_FUNCTION(
   (int), // the return type of the function, the parentheses are required.
   function_with_named_parameters, // the name of the function.
   tag, // part of the deep magic. If you use BOOST_PARAMETER_NAME you need to put "tag" here.
   (required // names and types of all required parameters, parentheses are required.
      (foo, (int))
      (bar, (float))
   )
   (optional // names, types, and default values of all optional parameters.
      (baz, (bool) , false)
      (bonk, (std::string), "default value")
   )
)
{
   if (baz && (bar > 1.0)) return foo;
      return bonk.size();
}

//client code
function_with_named_parameters(1, 10.0);
function_with_named_parameters(7, _bar = 3.14);
function_with_named_parameters( _bar = 0.0, _foo = 42);
function_with_named_parameters( _bar = 2.5, _bonk= "Hello", _foo = 9);
function_with_named_parameters(9, 2.5, true, "Hello");

Modern named parameters

Modern C++ opened some new doors. Can the new language features lead to slimmer implementations of named parameters?

Lambdas

Method chaining is verbose. I don’t like adding all the functions returning the object itself. What about defining just a struct and assign all the members through a lambda?

struct FileRecipe
{
   string Path; // mandatory
   bool ReadOnly = true; // optional
   bool CreateIfNotExist = false; // optional
   // ...
};

class File
 {
   File(string _path, bool _readOnly, bool _createIfNotexist)
      : path(move(_path)), readOnly(_readOnly), createIfNotExist(_createIfNotExist)
 {}

private:
   string path;
   bool readOnly;
   bool createIfNotExist;
 };

auto file =  CreateFile( "path", [](auto& r) { // sort of factory
   r.CreateIfNotExist = true;
});

You still have to provide a parameter object but this approach scales quite better than the classical named parameter idiom in which even chaining functions have to be written.

A variant consists in making File constructible from a FileRecipe (like the named parameter idiom).

How to improve the fluency of mandatory parameters? Let’s mix this approach with tags:

auto file =  CreateFile( _path, "path", [](auto& r) {
   r.CreateIfNotExist = true;
 });

But they are still positional. If you rest easy with a runtime error if a mandatory parameter is missing then use an optional type and check it at runtime.

CreateFile is trivial and left to the reader.

I’ve recently used this approach to configure test recipes and mocks. For example, I needed to create tests of a trivial dice game. Every game had a configuration and tests used to look like:

TEST_F(SomeDiceGameConfig, JustTwoTurnsGame)
{
   GameConfiguration gameConfig { 5u, 6, 2u };
}

By using the approach above we could have:

TEST_F(SomeDiceGameConfig, JustTwoTurnsGame)
{
   auto gameConfig = CreateGameConfig( [](auto& r) {
       r.NumberOfDice = 5u;
       r.MaxDiceValue = 6;
       r.NumberOfTurns = 2u;
   });
}

Diehards may suggest a macro to reduce verbosity:


TEST_F(SomeDiceGameConfig, JustTwoTurnsGame)
{
   auto gameConfig = CREATE_CONFIG(
       r.NumberOfDice = 5u;
       r.MaxDiceValue = 6;
       r.NumberOfTurns = 2u;
   );
}

Exploiting Variadics

Variadics can improve techniques I described above. What about Andrej’s tags approach? Tags could be preferred over the lambda + parameter object because you don’t have to create another object, you don’t have problems with settability and you consider all the parameters the same (e.g. by using the lambda approach you have to treat mandatory parameters differently). But I think tags would be better, if I could:

  • define only one overload of my constructor (or function),
  • decide the order of the parameters (pairs tag-value),
  • the two above + having optional and mandatory parameters.

Something simple like:

File f { _readonly, true, _path, "some path" };

or (my preference):

File f { by_name, Args&&... args) {} 

My idea is: I just want to use variadics to let the user decide the order and let her omit optional parameters.

Imagine two constructors:

File(string path, bool readonly, bool createIfNotExist) {} // all mandatory

template<typename... Args>
File(by_name_t, Args&&... args) {}

A instance of File can be created by using both. If you use the variadic one then I’ll look for all parameters in the pack and delegates the other constructor to really make the instance. Search is (at compile-time) linear over the pack that contains parameters in the order chosen by the caller.

[Note: my implementation is just a proof of concept I did more than one year ago (I only added decltype(auto) somewhere). It could be done better and better.]

Here is how the class designer may look at her class:

File(string path, bool readonly, bool createIfNotExists /*...*/)
   : _path (move(path)), _createIfNotExist(createIfNotExist), _readonly(readonly) // ,etc...
{
}

template<typename Args...>
File(named_tag, Args&&... args)
   : File{ REQUIRED(path), OPTIONAL(read, false) // , etc... } // delegating
{
}

Prior to show you a working code, it’s clear we can apply the same idea to proxies and obtain:

auto f = File { by_name, readonly=true, path="path" };

The real difference here is about forwarding: with proxies, we benefit from the syntax sugar (the operator=) but now we have to store the values and forward them (not ideal for non-movable/copyable types – and other problems could arise).

Here you can play with the code (and here is the same file on Gist). I first started with the tag version and then I tried with proxies. For this reason there are two versions: the former works with tags ([tag, value]…) and the latter with proxies ( [tag=value]…). Some code could (and should) be refactored.

You’ll find two sections called “PACK UTILS” (two versions: tag and proxy). These contain code I wanted to play originally (e.g. playing with variadics). I also think these kind of operations can be done by using std::forward_as_tuple and then by exploiting tuple’s utilities.

Another part of the code contains macros to retrieve parameters and to generate tags.

The final section is a full example.

Here is what a class looks like:

class window
{
public:
    // classic constructor
    window( string pTitle, int pH, int pW,
    int pPosx, int pPosy, int& pHandle)
       : title(move(pTitle)), h(pH), w(pW), posx(pPosx), posy(pPosy), handle(pHandle)
    {
    }

    // constructor using proxies (e.g. _title = "title")
    template<typename... pack>
    window(use_named_t, pack&&... _pack)
       : window { REQUIRED_NAME(title), // required
                  OPTIONAL_NAME(h, 100), // optional
                  OPTIONAL_NAME(w, 400), // optional
                  OPTIONAL_NAME(posx, 0), // optional
                  OPTIONAL_NAME(posy, 0), // optional
                  REQUIRED_NAME(handle) } // required
    {
    }

    // constructor using tags (e.g. __title, "title")
    template<typename... pack>
    window(use_tags_t, pack&&... _pack)
       : window { REQUIRED_TAG(title), // required
                  OPTIONAL_TAG(h, 100), // optional
                  OPTIONAL_TAG(w, 400), // optional
                  OPTIONAL_TAG(posx, 0), // optional
                  OPTIONAL_TAG(posy, 0), // optional
                  REQUIRED_TAG(handle) } // required
    {
    }

private:
  string title;
  int h, w;
  int posx, posy;
  int& handle;
};

You see, both named and tag constructors always delegate the real constructor to perform initialization.

The following code fragment shows how the caller uses the contraption:

int i=5;
// tags version
window w1 {use_tags, __title, "Title", __h, 10, __w, 100, __handle, i};
cout << w1 << endl;

// proxies version
window w2 {use_named, _h = 10, _title = "Title", _handle = i, _w = 100};
cout << w2 << endl;

// classic version
window w3 {"Title", 10, 400, 0, 0, i};
cout << w3 << endl;

by_name here is called use_named, but the meaning is the same.

Pros:

  • mandatory and optional parameters are uniform (named or tagged)
  • order is not defined a priori
  • tag approach has no forwarding issues

Cons:

  • compile-time errors could be hard to understand (static_assert helps a bit)
  • available parameters should be documented
  • pollution of the namespace scope still remains
  • default-values are always evaluated (some improvements for laziness are possible)
  • proxy approach is not ideal for forwarding.

A note about the first trouble: Clang is a gentleman and it complains politely. For instance, suppose I forget a title for my window. Here is the output:

main.cpp:28:2: error: static_assert failed "Required parameter"
        static_assert(pos >= 0, "Required parameter");
        ^             ~~~~~~~~
main.cpp:217:14: note: in instantiation of template class 'get_at<-1, 0>' requested here
                :       window { REQUIRED_NAME(title),
                                 ^

This way you know precisely where you miss a required parameter. This could be improved.

A minimalistic approach using std::tuple

[Note: completely by Davide Di Gennaro]

We can exploit some of the power of std::tuple to write an extremely compact and portable implementation. We will stick to some simple principles:

  • the parameter pack will be a special tuple, where a “tag type” is immediately followed by its value (so the type would be something like std::tuple<age_tag, int, name_tag, string, … >)
  • the standard library already has utility functions to forward / concatenate objects and tuples that guarantee optimal performance and correctness in move semantics
  • we will use a macro to introduce global constants that represent a tag
  • the syntax for constructing a parameter pack will be (tag1=value1)+(tag2=value2)+…
  • the client will take a parameter pack as a reference to template type, e.g.
template <typename pack_t>
void MyFunction([whatever], T& parameter_pack)      // or const T&, T&&, etc.
  • With a function call, the client will extract a value from the pack and (say) move it into a local variable.

Ideally, here’s how the code will look like:

namespace tag
{
   CREATE_TAG(age, int);
   CREATE_TAG(name, std::string);
}

template <typename pack_t>
void MyFunction(T& parameter_pack)
{
   int myage;
   std::string myname;
   bool b1 = extract_from_pack(tag::name, myname, parameter_pack);
   bool b2 = extract_from_pack(tag::age, myage, parameter_pack);
   assert(b1 && myname == "John");
   assert(b2 && myage == 18);
}

int main()
{
   auto pack =  (tag::age=18)+(tag::name="John");
   MyFunction(pack);
}

Here is how the implementation may look like. We will omit most of the potential optimizations for sake of clarity (and they are probably unnecessary).

First, the macro:

#include <tuple>
#include <utility>

template <typename T>
struct parameter {};

#define CREATE_TAG(name, TYPE) \
\
   struct name##_t \
   { \
      std::tuple<parameter<name##_t>, TYPE> operator=(TYPE&& x) const \
      {  return std::forward_as_tuple(parameter<name##_t>(), x); } \
      \
      name##_t(int) {} \
}; \
\
const name##_t name = 0

The expansion of CREATE_TAG(age, int); creates a class and a global object. Note that this will work if positioned inside a namespace.

struct age_t
{
   std::tuple<parameter<age_t>, int> operator=(int&& x) const
   {
      return std::forward_as_tuple(parameter<age_t>(), x);
   }
   age_t(int) {}
};

const age_t age = 0;

 

Conceptually the assignment

age = 18

Translates into something similar to:

make_tuple(parameter<age_t>(), 18);

Observe that we wrote:

std::tuple<parameter<age_t>, int> operator=(int&& x) const

As written, we require an r-value on the right. First, this is an extra safety feature: to increase the readability of the code with parameter packs, you may want to assign constants, not variables (otherwise, renaming the variable would be sufficient). e.g.

int myage = 18;
f(myage); // ok, clear

g((...) + (age=18)); // ok, clear
g((...) + (age=myage)); // compiler error, and redundant from a readability point of view

Second, we can exploit move semantics:

The difference between

std::tuple<parameter<age_t>, int> operator=(int&& x) const
{
   return std::make_tuple(parameter<age_t>(), x);
}

and


std::tuple<parameter<age_t>, int> operator=(int&& x) const
{
   return std::forward_as_tuple(parameter<age_t>(), x);
}

is very subtle. The latter returns std::tuple<…, int&&>, but since the return type is tuple<…, int> then tuple’s move constructor is invoked.
Alternatively we could write


std::tuple<parameter<age_t>, int> operator=(int&& x) const
{
   return std::make_tuple(parameter<age_t>(), std::move(x));
}

Now, we add a suitable tuple-concatenation operator.

We informally agree that all tuples starting with parameter<T> have been generated by our code, so without any explicit validation, we just cat them:

template <typename TAG1, typename... P1, typename TAG2, typename... P2>
std::tuple<parameter<TAG1>, P1..., parameter<TAG2>, P2...>
operator+ (std::tuple<parameter<TAG1>, P1...>&& pack1, std::tuple<parameter<TAG2>, P2...>&& pack2)
{
    return std::tuple_cat(pack1, pack2);
}

Very simply, this function will do a simple pattern matching on two tuples: if they both look like:

tuple<parameter<tag>, type, [maybe something else]>

then they are joined together.

Finally, we publish a function to perform the extraction of an argument from the pack. Note that this function has move semantics (i.e. after a parameter is moved out of the pack).

template <typename TAG, typename T, typename... P, typename TAG1>
bool extract_from_pack(TAG tag, T& var, std::tuple<parameter<TAG1>, P...>& pack);

the effect of this function is:

if the “pack” contains parameter<TAG>, then var receives the value immediately following, and the function returns true. otherwise something bad happens (we can choose between: a compiler error, return false, throw exception, and a few more…)

To make this selection possible, actually the function will be:

template <typename ERR, typename TAG, typename T, typename... P, typename TAG1>
bool extract_from_pack(TAG tag, T& var, std::tuple<parameter<TAG1>, P...>& pack)

So we will invoke it as:

extract_from_pack< erorr_policy > (age, myage, mypack);

Due to variadic templates pattern matching, extract_from_pack knows that the pack has the form tuple<parameter<TAG1>, … > so it needs to examine recursively if TAG is equal to TAG1. We will do this dispatching the call to a class:

extract_from_pack< erorr_policy > (age, myage, mypack);

calls

extractor<0, erorr_policy >::extract (age, myage, mypack);

which in turn calls

extractor<0, erorr_policy >::extract (age, myage, std::get<0>(pack), mypack);

which has two overloads:

extract(TAG, … , TAG, …)

which succeeds, performs the assignment and returns true, or

extract(TAG, … , DIFFERENT_TAG, …)

which keeps on iterating, calling again

extractor<2, erorr_policy >::extract (age, myage, mypack);

when iteration is not possible, error_policy::err(…) is invoked.


template <size_t N, typename ERR>
struct extractor
{
   template <typename USERTAG, typename T, typename TAG, typename... P>
   static bool extract(USERTAG tag, T& var, std::tuple<parameter<TAG>, P...>&& pack)
   {
      return extract(tag, var, std::get<N>(pack), std::move(pack));
   }

   template <typename USERTAG, typename T, typename TAG, typename... P>
   static bool extract(USERTAG tag, T& var, parameter<TAG> p0, std::tuple<P...>&& pack)
   {
      return extractor<(N+2 >= sizeof...(P)) ? size_t(-1) : N+2, ERR>::extract(tag, var, std::move(pack));
   }

   template <typename USERTAG, typename T, typename... P>
   static bool extract(USERTAG tag, T& var, parameter<USERTAG>, std::tuple<P...>&& pack)
   {
      var = std::move(std::get<N+1>(pack));
      return true;
   }
};

template <typename ERR>
struct extractor<size_t(-1), ERR>
{
   template <typename TAG, typename T, typename DIFFERENT_TAG, typename... P>
   static bool extract(TAG tag, T& var, std::tuple<parameter<DIFFERENT_TAG>, P...>&& pack)
   { return ERR::err(tag); }
};

template <typename ERR, typename TAG, typename T, typename... P, typename TAG1>
bool extract_from_pack(TAG tag, T& var, std::tuple<parameter<TAG1>, P...>& pack)
{
   return extractor<0, ERR>::extract(tag, var, std::move(pack));
}

Due to the flexible nature of parameter packs, the best error policy would be a plain “return false” (any stronger error would in fact make that parameter mandatory). So:

struct soft_error
{
   template <typename T>
   static bool err(T)
   {
      return false;
   }
};

However we are free to chose any of these:

struct hard_error
{
   template <typename T>
   static bool err(T); // note that static_assert(false) here won’t work. can you guess why?
};

struct throw_exception
{
   template <typename T>
   static bool err(T)
   {
      throw T();
      return false;
   }
};

An additional improvement could be a redundancy check, that prevents code like (age=18)+(age=19).

The code for this is short, but it requires some subtle manipulation with variadic templates, so we leave it as an excercise.

Final notes

I have not discussed about runtime techniques, e.g.:


void MyFunction ( option_parser& pack )
{
   auto name = pack.require("name").as<string>();
   auto age = pack.optional("age", []{ return 10; }).as<int>();
   ...
}

Whereas the opening techniques I have presented are fairly consolidated, last ideas are just work in progress and I’m still trying to understand if they make sense. The code I’ve shown is just a proof of concept and it does not claim to be optimal nor suitable for production. I wrote these lines more than one year ago, to practice with variadics and I finally found some time to summarize my ideas in a decent (I hope) post and share with you. If this will be helpful or inspiring to anyone, I’ll be really glad.

I found a recent proposal aiming to introduce named arguments in C++ here. Cool!

Anyhow I have a question for you: where would you have wanted to use named parameters in C++?

I read the article “Enforcing the Rule of Zero” from latest Overload (of ACCU) and I’d like to point out something that I misapprehended at a first reading.

I’m not explaining what the Rule of Zero is. If you’ve never heard about it, I heartily suggest you to read Martinho Fernandes’ article. The rule states (basically): classes that have custom destructors, copy/move constructors or copy/move assignment operators should deal exclusively with ownership. This rule is an application of the Single Responsability Principle.

Back to the article, a clever point the author underlines is about polymorphic deletion: what to do when we want to support polymorphic deletion, or when our classes have virtual functions? Quoting Herb Sutter’s: If A is intended to be used as a base class, and if callers should be able to destroy polymorphically, then make A::~A public and virtual. Otherwise make it protected (and not-virtual).

For example:


struct A
{
   virtual void foo();
};

struct B : A
{
   void foo() override ;
   int i;
};

A* a = new B{}; 

delete a; // ops, undefined behavior

To correctly destroy B, A should have a virtual destructor:


struct A
{
   virtual ~A() {}
   virtual void foo();
};

Problem: now the compiler won’t automatically generate move operations for A. And, worse, in C++14 this deficiency is extended to copy operations as well. So, the following may solve all the problems:


struct A
{
   //A() = default; // if needed
   virtual ~A() = default;
   A(const A&)=default;
   A& operator=(const A&)=default;
   A(A&&)=default;
   A& operator=(A&&)=default;

   virtual void foo();
};

This is called the Rule of Five. The author then wisely proposes to follow the second part of the Rule of Zero, that is: “Use smart pointers & standard library classes for managing resources”. I add: or use a custom wrapper, if an STL’s class does not fit with your needs. The key point is: use abstraction, use RAII, use C++.

Then the author suggests to use a shared_ptr:

struct A
{
   virtual void foo() = 0;
};

struct B : A
{
   void foo() override {}
}

shared_ptr<A> ptr = make_shared<B>(); // fine

This works well and it avoids the (more verbose) Rule of Five.

Why shared_ptr and not simply unique_ptr? Let me remark the key point: the article never says “use any smart pointer”, because not every smart pointer would do. In fact a plain unique_ptr would not have worked.

One of the (many) differences between unique and shared pointers is the deleter. Both can have a custom deleter, but in  unique_ptr the deleter is part of the type signature (namely, unique_ptr<T, Deleter>) and for shared_ptr is not: shared_ptr has a type-erased deleter (and in fact also a type-erased allocator).

This implies that shared_ptr<B> has a deleter which internally remembers that the real type is B.

Consider the example I borrowed from the article: when make_shared<B> is invoked, a shared_ptr<B> is constructed as expected. shared_ptr<B> constructs a deleter which will delete the B*. Later, shared_ptr<B> is passed to shared_ptr<A>’s constructor: since A* and B* are compatible pointers, shared_ptr<B>’s deleter is “shared” as well. So even if the type of the object is  shared_ptr<A>, its deleter still remembers to delete a pointer of type B*.

Conversely, unique_ptr<T> has a default deleter of type std::default_deleter<T>. This is because the unique_ptr is intended to be use exactly as a light wrapper of a delete operation (with unique ownership – and not merely a scoped one). std::default_deleter<A> can be constructed from std::default_deleter<B> (since pointers are compatible), but it will delete an A*This is by design, since unique_ptr is intended to mimic closely operator new and delete, and the (buggy) code { A* p = new B; delete p; } will call delete(A*).

A possible way to work around this issue is to define a custom type-erased deleter for unique_ptr. We have several ways to implement this. One uses std::function and the other uses a technique described by Davide Di Gennaro in his book Advanced C++ Metaprogramming, called Trampolines. The idea for the latter was suggested by Davide, in a private conversation.

Using std::function is the simplest way:


template<typename T>
struct concrete_deleter
{
   void operator()(void* ptr) const
   {
      delete static_cast<T*>(ptr);
   }
};

...

unique_ptr<A, function<void(void*)> ptr { new B{}, concrete_deleter<B>{} };

Here we are using type-erasure, again. std::function<void(void*)> is a wrapper to any callable object supporting operator()(void*). When the unique_ptr has to be destroyed it calls the deleter, that is actually a concrete_deleter<B>. Internally, concrete_deleter<T> casts the void* to T*. To reduce verbosity and avoid errors like { new B(), concrete_deleter<A>{} }, you can write a make_unique-like factory.

The second solution is cunning and it implements type-erasure without a virtual function call (an optimization which goes beyond what std::function can really use):

struct concrete_deleter
{
   using del_t = void(*)(void*);
   del_t del_;

   template <typename T>
   static void delete_it(void *p)
   {
      delete static_cast<T*>(p);
   }

   template<typename T>
   concrete_deleter(T*)
     : del_(&delete_it<T>)
   {}

   void operator()(void* ptr) const
   {
     (*del_)(ptr);
   }
};
...

template<typename T, typename... Args>
auto make_unique_poly(Args&&... args)
{
   return unique_ptr<T, concrete_deleter>{new T{forward<Args>(args)...}, static_cast<T*>(nullptr)};
}

...
unique_ptr<A, concrete_deleter> ptr = make_unique_poly<B>();

The idea is storing the type information in the del_ function pointer, directly.

[Edit]

As many readers suggested, this can be done also by using a lambda. This way we get rid of the concrete_deleter support struct. I’m just afraid of this solution (that was in the first draft of this post) because if you use a generic type like the following:

unique_ptr<A, void(*)(void*)>

When you read the code you don’t know, at a first sight, what unique_ptr means. Worse, you may re-assign the unique_ptr to another one, passing a totally different lambda that observes the same signature.

Moreover, as Nicolas Silvagni commented, the size of unique_ptr<A, concrete_deleter> (or the one using a lambda) is greater than unique_ptr<A> (typically doubles – e.g. 8 bytes vs 16 bytes, on 64bit architecture). To prevent this, an intrusive approach is possible (read the comments for details). Alas, an intrusive approach does not follow the design of unique_ptr (and of STL wrappers in general) that is non-intrusive.

[/Edit]

So to sum up, here are the possible workarounds:

  1. Use shared_ptr (if possibile),
  2. Apply the Rule of Five (so declare a virtual destructor),
  3. Use unique_ptr with a custom deleter.

 

What do you think?

Acknowledgments

Many thanks to Davide Di Gennaro for reviewing this article and suggesting me some improvements. Some ideas arised from a private conversation we had.

A couple of weeks ago, a friend of mine coded an output stream that outputted strings through Win32 OutputDebugString function (you can use DebugView to monitor these kind of traces). His implementation had two main problems:

  • it was designed quite poorly (as you’ll see in a while),
  • it didn’t allow a real formatting.

With the second point I mean: everytime operator<<(stream&, something) was called, something was sent to OutputDebugString. I paste here a facsimile of his code:


class debug_stream : public std::ostringstream
{
public:
    template<typename T>
    friend debug_stream& operator<<(debug_stream& os, T&& s);
};

template<typename T>
debug_stream& operator<<(debug_stream& os, T&& s)
{
    (ostringstream&amp;)os << s;
    PrintToDebug(os.str());
    os.str("");
    return os;
}

//...
debug_stream dbgview;
dbgview << "This is a string" // sent to DebugView
        << 10.01 // sent to DebugView, separately
        << ...
        endl;

What I mostly dislike of this code is the design choice to inherit from std::ostringstream, being forced to override operator<< as the only (simple) option to print something to the DebugView. This makes things even more difficult when you have to preserve formatting rules. The preferable choice would be storing stuff until they need to be outputted to DebugView (e.g. when std::endl is used).

I suggested him to change his point of view and thinking of the real nature of streams: streams are serial interfaces to some storage. That is, streams are just a way to send some data (e.g. chars) to devices (e.g. files), with a common interface (e.g. operator<<). In this example, we can think of DebugView as another device and not as another stream. In the standard library, these devices are called streambuffers (the base class is std::streambuf) and each one takes care of the various issues specific to that kind of device. The simplest streambuffer you can imagine is an array of characters. And if we are lucky, the standard library already provides some std::streambuf implementations we can take advantage of.

Let me recap the situation, with simple (and not so accurate) words:

  • we want to output data through different devices (e.g. files, sockets, console, …),
  • C++ provides a common interface to output data to devices, that is the concept of (output) stream,
  • streams need to be decoupled from devices. They don’t know details like specific synchronization issues,
  • buffers handles specific issues of devices.

Streams can be seen as a FIFO data structure, whereas buffers (that contain raw data) provide random access (like an array).

Let’s turn back to my friend’s problem:  roughly, he just has to wrap a sequence of characters and send it to DebugView as soon as std::endl is used. For example:


dbgview << "Formatted string with numbers " << 2 << " and " << setprecision(3) << 10.001 << endl;

This means he just needs a way to modify how the buffer “synchronizes” with the “underlying device” (that is: the buffer stores some characters and at some point it has to write them to its “target” – e.g. a file, the console or the DebugView). Yes, because if the standard library already provides a stream buffer that just maintains a sequence of characters, he doesn’t need to alter that mechanism at all. And he is lucky because C++ has the std::stringbuf, right for him!

So the idea is to inherit from std::stringbuf and let it do the most of the job. We only need to change the way our buffer writes the buffered data (aka: the formatted string) to the target (aka: DebugView). And this is a job for streambuf‘s sync() virtual function, that is called, for example, when std::endl manipulator is used on the stream using the buffer. Cool!

This is the simplest code I thought about and I sent to my friend:

#include <sstream>
#include <windows.h>

class dbgview_buffer : public std::stringbuf
{
public:
    ~dbgview_buffer()
    {
       sync(); // can be avoided
    }

    int sync()
    {
        OutputDebugString(str().c_str());
        str("");
        return 0;
    }
};

Two notes:

  • I call sync() in the destructor because the buffer could contain some data when it dies (e.g. someone forgot to flush the stream). Yes, this can throw an exception (both str() and OutputDebugString could throw), so you can avoid this choice,
  • I clear the current content of the buffer after I send it to DebugView (str(“”)).

As you suspect, str() gives you the buffered std::string. It has also a second form that sets the contents of the stream buffer to a passed string, discarding any previous contents.

So you can finally use this buffer:

dbgview_buffer buf;
ostream dbgview(&buf);
dbgview << "Formatted string with numbers " << 2 << " and " << setprecision(3) << 10.001 << endl;
// only one string is sent to the DebugView, as wanted

std::streambuf (and its base class std::streambuf) handles the most of the job, maintaining the sequence of characters. When we use operator<< it updates this sequence (dealing also with potential end-of-buffer issuess – see, for instance, overflow). Finally, when we need to write data (e.g. synchronize with the underlying device) here it comes our (pretty simple) work.

Clearly you can also derive from ostream, hiding the buffer completely:

class dbgview_t : public ostream
{
public:
    dbgview_t() : ostream(&m_buf)
    {}
private:
    dbgview_buffer m_buf;
};

// we can also declare a global dbgview, like cout/cin/cerr:
extern dbgview_t dbgview; // defined elsewhere

The moral of this story is: sometimes, we have to think of what a component is made of. Can I just change one of its parts? If you want a faster car maybe you can just replace its engine rather than imagining to buy a new car! STL are quite extensible and often it happens that you can “configure” one of its classes just replacing a collaborator of it. This is quite related to the Open/Closed Principle, that is “software entities should be open for extension, but closed for modification“. C++ has different ways to preserve and pursue this concept, like object orientation and generic programming. And you can also combine them!

Being part of a team could be cool and istructive, and I’m not referring only to software development. For example, I’ve played volleyball for 13 years, in 6 different teams and I’ve enjoyed a lot at playing with different people with different styles. Sometimes people forget that you can learn not only from the best players, but also from the worst ones.

From the best you can acquire tips and tricks, pills of experience, you can refine your technique, you can just mimic him (or at least try!). From the worst player you may understand how to deal with failures and recurrent errors. The worst player should be (I repeat, should be) the athlete who gives her best to improve herself. This desire to get better is instructive and, often, is not very apparent in high level players. Anyhow, each member of a team can learn and teach at the same time. I consider this thing very valuable.

Same concepts apply to work, for example in a team of software developers. Team is first about people and only after about software, volleyball or whatever you want. So a team of developers can be messed up by the same problems of a team of footballers. Generally these problems are some difficult people and situation. Here I’m trying to describe four types of problematic teammates, basing this classification on my experience and lots of discussions with other software developers.

These guys are initially welcomed into the team, but are capable of causing troubles when their shortcomings become apparent. And I’m not talking of technical skills, because those with insufficient technical skills are usually not hired (usually…). I’m talking about attitude, that is more difficult to treat.

Let’s start.

Rambo is the first type of problematic programmer. He is talented and everybody trusts and depends on him. If you can’t solve a problem Rambo is your guy. Rambo knows he is the best, he enjoys his position and sometimes (if the management is too weak) he also keeps the top-level manager in check. In fact, the top-level manager invested him of a special power. In the worst case, Rambo takes complete ownership of his code and tries not to let anyone else near it. He doesn’t trust anyone, so no one can touch his code. And often it happens that Rambo voluntarily write difficult code, so no one can (easily) understand it.

Another bad thing that can happen is when your team builds and evolves a product. Rambo can be assigned to make the new stuff, because he is the best and he knows how to do them fastly. Also clients trust him. This can lead also to jealousy inside the team.

Rambo has a high contractual power specially if the top-level manager relies heavily on him. It is useless to say which troubles the whole team may experience if Rambo quits, leaving a vast quantity of source code that is difficult to deal with. And what if he fails or he is sick? He is a bottleneck. The bad news is that Rambo is mostly a tricky management problem. Why tricky? Because it can happen that the team-leader cannot really “order” anything to Rambo, in fact he responds only to the top-manager.

The Prima Donna is another plague. He is the best and he knows it. If you criticize him, you’ll suffer his wrath! His idea is the best, he’s been a programmer since he was a child, his ideas are always excellent and his code is perfect ever. He’s not capable to take criticism. But unlike Rambo, he was not invested of a special power by the top-level manager so – from a management standpoint – the prima donna is a team-leader problem. The prima donna lacks interpersonal skills and he divides other teammates in two classes: those who are less skilled than he and threats. He generally criticizes who he perceives as a threat by insulting their intelligence and their skills.

Very likely, the prima donna is the biggest pain for a team. He can cause other skilled – but less arrogant – people to quit. Sometimes the prima donna forms an alliance with Rambo (or at least he tries). And if he succeds the things will become even worse.

The parasite is the factotum but the master of none. He knows all about the system (the product), so he is one of the most important “historical mind” of the team. Database, GUI, core, … he has worked on every aspect of the system so when you have problems on old (and very used) features then he is your man. Sometimes he just wants to preserve his job so he is not interested to innovate the system with new technologies, that – from his point of view – are something new to learn or, even worse, something that new programmers can learn easily.

The parasite is the most useful of the negative developer types and, sometimes, can be viewed as positive. He is not a menace for the team, indeed, in some cases he is the developer that is worth preserving. The good news is that the team-leader can easily make people working with him so his know-how flows in the team.

The coward programmer is the typical computer geek. He is personality-deficient even if he is very capable. He tends to be very withdrawn and taking criticism is also difficult for him. In fact, he considers a critics like a reproach because his communication model is freezed at the age of 12. Due to his awe and introversion, it’s not unusual that he considers sending emails the best way of communicating.

The real problem is that he never really participates the team. He doesn’t propose his – often excellent – ideas. This lack of communication is painful and the company doesn’t get all the real value of this kind of programmer. A coward programmer is hard to deal with for the team-leader as well as for the top-level manager, because in the coward’s mind they are too high level to communicate peacefully with him. So, each teammate (seen like a “peer”) should help the coward to improve his communication skills, for example by integrating him a bit more in the job. Also pair-programming may help.

So I presented four kind of painful teammates. A thing that sometimes I stressed was about top-level management and team-leader. They are responsible to identify and deal with these people from a management standpoint. Obviously, if you are a teammate of Rambo or Prima Donna, you have to deal with them from a technical and often interpersonal point of view. Suggestions are hard to make. People are different and each character has subjective reaction.

If you have to work with a prima donna, the tip I’d give you is to never react to his critiques with another critique. Be professional, the battlefield of a prima donna is about provocations and technical religion. If you like Java and he hates Java, don’t try to convince him that Java can be used for a project. Rather, say “ok, you’re right, let’s talk about our job now”. And if it is impossible for you to work with him, talk to your team-leader.

As I wrote, sometimes Rambo leads to jealousy inside the team because he is assigned to new and cool stuff. If your company organizes meeting with the top-level manager (individual or group), tell you want to join new projects. Try to be pragmatic, convince him you have the required skills. This can be useless, but trying is better than nothing.

Parassites and coward programmers are generally harmless from an interpersonal standpoint. You could be in contrast with an “old” parassite who does not like innovation. Here, again, be pragmatic, make clear the benefits of the new stuff. But don’t claim to win every single battle!

And what if you identify yourself in one of this category?! Fuck you. No, I’m just joking:) Well! Consider you could be a sort of problem for your teammates and for your company. And if you don’t care maybe other people do!