C++ performance guidelines

Optimization barriers are those constructs that the compiler cannot or does not optimize. It might also be that they prevent only certain types of optimizations.

The first category of optimization barrier is made by those constructs that have visible side effects, like

The second category is made of constructs that are outside of the C++ language. These are mainly externally compiled files, like object files, shared libraries, and so on.

The third category is made of constructs that the compiler is not good at optimizing:

A sufficiently smart compiler can overcome many limitations like assembly, exceptions, memory barriers, and some system calls like memory allocation. A conforming compiler might even modify a bubble-sort to quick-sort, as long as the end result is the same. This is the reason why the correctness of the program should not rely too much on optimization. Imagine if you update the compiler and the performance is awful as your bubble-sort is not optimized anymore to a quick-sort, and you did not even know in the first place that it has been optimized so aggressively.

While most optimizations barrier/side effects are generally needed (somewhen the program needs to synchronize between threads or output something), it is generally best if those operations can be gathered together.

For example

vs

or

vs

For the same reason, try not to use a mutex recursively, thus use std::mutex instead of std::recursive_mutex. It helps to make your class both simple and more performant. A sufficiently smart compiler could optimize both classes to produce the same code, I bet patches to the corresponding compiler teams are welcome.

It’s useful for using it as a guideline when writing code

According to this guideline, reading from memory is much faster than allocating (normally a system call).

Indeed changing

to something like

is faster.

One could measure if replacing a + b + c with concat is worth it. Unless the application is concatenating a lot of string, the performance gain might not be noticeable. But as the time at disposal is limited, using consistently something like concat has no disadvantage, and can even be more beneficial as shown in the benchmark if some of the concatenated strings are not std::string, but std::string_view, literals, or other types. It also has the advantage that the performance gain might be noticeable on less powerful platforms, or when using tools like Valgrind, sanitizers, or when fuzzing.

Defining constants seems to be something so easy…​ and yet it is a can of worms. There is the issue with visibility, the initialization order fiasco, and the fact that constants (even unused) that allocate memory (like std::string) won’t free it until the program is exiting.

Thus if possible, just make all constants compile-time constant, ie use constexpr. Avoid allocating owning types, as they need to copy unnecessarily the content. Writing a constexpr and efficient container, like map is not an easy task. But if the sole reason to exist for this class is to hold compile-time constants, we can take some shortcuts:

The easiest implementation is just using an array of pairs as the underlying data structure. Something like the following ct_map class should be a good starting point

By taking the call to std::sort out, it can be backported to C++11 with minor changes if one backports std::index_sequence too.

Indeed, for a "small" set of values it outperforms the map containers from the standard library. Actually, it outperforms std::map even with optimization disabled, but not std::unordered_map.

The allocation, which would happen only once at startup, is not taken into account, but creating a std::*map is not trivial. Creating a constant std::map/std::unordered_map is approximately 450 lines of assembly (100 for the data), thus 350 lines of logic for creating the data structure. Creating the constexpr ct_map is just the data.

Thus while counting assembly lines is not a good performance metric, in this case, it shows how much difference defining a simpler data structure can reduce code bloat. Unless we are starting the same program a lot of times (for example when fuzzing…​), no one might notice the difference. On the other hand, limiting the size of the executable could have nice side-effects, smaller updates can, for example, be downloaded and installed faster.

Those and others are multiple reasons why I would prefer ct_map over std::*map, even if from a strict performance view we do not have necessarily always a gain when storing constants.

Singletons are the way to initialize global data at runtime in C++. The initialization is even thread-safe, but we might not always need it. Either we do not have multiple threads accessing the singleton, or we are already using something else to ensure consistency with multiple threads.

In both cases, even if small, there is an overhead for accessing the data, which would not be there if we simply made it global.

For constants, true constants, use constexpr, but if not possible continue to use singletons, especially because they avoid the init-order fiasco.

Passing owning types might require allocation even if passed by reference.

Types that do not own the content, like std::string_view, and std::span, are "small". They are conceptually a couple of pointers (or a pointer and a length), and thus should be passed by value, and not const-reference.

View types are also incredibly useful for defining constants. The class ct_map has the disadvantage that the size is part of the type. This makes it impossible to store, for example, multiple ct_map with different sizes in an array and limits the possibility to pass it as a parameter to functions.

On the other hand, a type like string_view, does not have such limitations, as the size is not part of the type, but can still be queried at compile-time.

This only applies if the move constructor is efficient. It generally is, for example for std::string, std::vector, …​, but there might be exceptions, especially for user-defined types, arrays, and thus classes like ct_map.

Passing ownership by value is also consistent with move-only types, like std::unique_ptr.

versus

As the move is cheap (possibly swapping values on the stack, and not copying the content on the heap of std::string), passing by value has more advantages than passing by const-reference.

Pass "big"/"complex" types by reference. Pass "small"/"simple" types by value.

Big and small depend on the context, but if void foo(int, int, int, int, int) is an acceptable function signature, then so should be struct p{int a; int b; int c; int d; int e;}; void foo(p);, as it has the same performance implications.

Complex types are those that have a non-default copy-constructor, like containers from the standard library.

When working with multiple threads, passing big types by value might be more performant than sharing them, and synchronizing access.

Also passing by value creates optimization opportunities that do not depend on inlining, like tail call optimizations.

Returning by value is the fastest way, with some care, to return a value from a function.

Until C++11 the language needed the definition of a copy-constructor which might have been expensive, but in practice, it did not need to be called, as RVO (return value optimization) was a thing at least already in 1996! I did not test with such old compilers if RVO could be forced though.

With C++11, thanks to move semantic, even if RVO cannot be applied or a class did not have a copy-constructor, the cases where returning by value was not an option shrunk drastically, as a move constructor is sufficient.

With C++17, which adds structured bindings and mandates RVO for some cases, I believe there are no more use cases where one should prefer one or multiple out-parameters, it does not matter how big or complex the data structure is.

Note: In-out parameters cover a different use case, so they should not be dismissed.

As mentioned previously, iterating multiple times over a linear data structure generally costs less than allocating memory.

Some containers of the standard library have a reserve method for preallocating enough memory. Since C++11, all containers from the standard library can be initialized directly, which ensures that memory will be claimed only once.

This logging interface has the disadvantage that it makes it hard to avoid unnecessary computations, as the caller should check each time if logging is enabled or not. Provide a "lazy interface" (either directly or wrap the functionality), there are many possibilities:

A possible wrapper could look like that:

Otherwise, the user needs to save all input parameters of the constructor for creating a copy, which is generally more costly.

Also for debugging being able to copy a value is useful, and for testing postconditions, it might be necessary to copy a value before mutating. Last but not least, copying can be useful for providing a stronger exception guarantee.

IO is slow compared to, for example, parsing something from a buffer.

std::set is, in my experience, overused, especially for small containers or constants. Creating a vector (or array), sorting it, and removing duplicates is generally faster, and also searching elements in a sorted std::vector is generally equivalent.

std::map preserves order, std::unordered_map is generally faster. An array of std::pair (like in ct_map) is also a viable alternative.

In general, avoid rewriting container classes, especially std::string. It has non-trivial optimizations like SSO, move semantic, amortized reallocation cost, …​ Thus prefer to use the standard library, but acknowledge that it does not cover efficiently all use-cases.

This should not be the main reason for choosing one algorithm over another. In many cases, it won’t give the correct solution, as cache locality plays, on modern hardware, a gigantic role.

Unless the data is very big (measure), everything else is secondary, unless the operations are really slow.

It also does not map very well on reality/physics, for example accessing an address in RAM is not constant time.

This is why, for example, std::vector should be almost always preferred over std::list.

I stumbled upon Optimized C++ by Kurt Guntheroth, Optimize String Use: A Case Study ^🗄️ and asked myself with the guidelines I wrote here, how my approach would compare to the proposed solutions, without changing the API too much (I ruled thus the solution that does not use std::string out).

The most easy-to-implement solution consists of a couple of lines of code (I did not use a lambda even if the book is from 2016):

It is surely simpler and easier to maintain than many proposed solutions, but is it performant?

Compared to remove_ctrl_block_args, remove_ctrl_erase_alg seems to be as fast. It would be faster if there were never elements to remove, but in general, there would be some.

remove_ctrl_erase_alg has two downsides: it copies the whole string, thus generally allocates more memory than necessary, and for deleting the characters it needs to shift them to the end, which might move a lot of them if the characters to be deleted are at the beginning.

copy_if_alg overcomes those limitations, even if it doubles the number of branches, and removes a trivial bulk copy operation. It also needs to iterate the original string twice, but as allocating are orders of magnitude slower, it should not be expected to be an issue, and indeed, this algorithm outperforms remove_ctrl_block_args, and all others presented in "Optimized C++" by Kurt Guntheroth.

There are other possible improvements, but without measuring those are not worth it, as they would make the code more complex/less maintainable and depend on the compiler’s ability to optimize code, on the input strings, on the runtime, and so on.

Use them for exceptional circumstances (unfortunately it’s hard to define what they are), as the exception path is not optimized much, but the non-exception path is generally more performant than testing for return codes.

There is no reason why the cold path could not be optimized more, it’s just that because of how exceptions are used (or disabled in some projects) there seems to be not that much interest in improving the situation.

noexcept, as optimization, makes little difference most of the time, even if it can help to remove dead code/remove hidden code paths. It also helps to reason about possible failures and about exception safety.

But for some functions, it is important to be noexcept from a performance perspective:

Notice that compiler-generated is already optimal, so do not implement those functions just for adding noexcept.

++it should be the preferred form when advancing.

In general, it does less work than the postfix operator (it++;). For trivial types (integer, pointer, …​) the compiler will generate the same code with minimal optimizations enabled, but for non-trivial types not necessarily. Using the prefix operator over the postfix operator for all types increases consistency, and avoids making it easy to overlook those types where the difference might be important.

When iterating over all elements, a for-each-loop (for(const auto& v : c){}) can also generate better code than a hand-made loop:

For example, instead of using integers as port numbers, define a structure.

A class port; can have a constructor that establishes an invariant (probably a range of valid values). In this case, instead of verifying in multiple routines if the passed parameter looks like a valid port number, the check needs to be implemented only once in the constructor.

Generally, this means avoiding doing the same test multiple times, which might be a performance boost. It also zeroes the possibility of using a non-validated port number and avoids having inconsistent tests.

The current design of the stream facilities (especially class inheritance), makes it slower than its C counterpart. Also, the interface is not as user-friendly. Many programmers use std::endl instead of '\n' for appending a newline, which flushes unnecessarily the buffer, and interleaved calls to the output stream are problematic. Also, internationalization is harder to achieve, and generally formatting output is a pain.

std::printf does not have all these drawbacks, but is very error-prone to use, and cannot be customized with other types.

Never facilities, like std::fmt and std::to_string, try not to repeat the same errors.

Also, SG16 considers using as the default output stream stdout and not std::cout for the new print facility ^🗄️.

Using std::puts for printing already formatted strings and avoiding stringstream for converting a number to strings, can not only increase performance but reduce compilation times and binary size too.