Features


Better Template Error Messages

Andrei Alexandrescu

We take for granted that debugging templates is a nightmare, but maybe that doesn't have to be true forever.


When templates were first incorporated into C++, perhaps few people realized how much power they would bring to the language. If we surveyed the last five years' worth of CUJ articles, we would turn up many more applications for templates than for polymorphism, one of the three pillars of object-oriented programming. But templates exact a price in their use, and it is one that causes many programmers to avoid using them altogether. Template code is hard to debug. In fact, it is so hard that one of our readers felt obliged to speak out about it. The article that follows is Andrei Alexandrescu's call for compiler vendors to improve the template diagnostics produced by their compilers.

We've given three compiler experts the opportunity to respond to Alexandrescu's proposal:

All three were gracious enough to offer their comments. We present their edited responses immediately after this article.

We would not claim that the solutions proposed here are the only possible ones, nor even the best. But it is our hope that this article will stir up discussion among compiler users and vendors. Templates are a powerful tool for writing safe, efficient, and reusable code. We think it's a shame that such a useful tool is marred by cryptic error messages. If you feel the same way, let your compiler vendors know. C++ is a great language. Let's help it live up to its full potential. — mb

Andrei Alexandrescu writes:

This is an open letter to compiler vendors and the C++ community. It presents a brief proposal that aims to make diagnostic messages generated by C++ compilers easier to read and understand in the presence of templates — including, of course, templates from the Standard C++ library.

The Problem

Two months ago, an angry colleague of mine learning STL emailed me an error message caused by the incorrect usage of an iterator object instead of a const_iterator object in a standard map container. The message was reasonable, if only you could read it. It was a kilobyte long. Seeing that, I started thinking of a convention that might lead to meaningful error messages in the presence of heavy use of templates.

During the recent C++ World Conference, I found out that many programmer teams and companies feel reluctant to start using the standard library — and templates in general — partly because of the incredibly messy error messages that result. (And you know what programming with STL is like. Once you get the program to compile, it will often work from the start, but getting it to compile is like pulling teeth.)

The problem stems from the fact that types obtained through template specializations tend to have very long names. Introducing typedefs, aggregating templates, and having defaults in the template argument list all make matters even worse. All such practices can lead to names hundreds of characters long, without the programmer having typed much. It's hard to know what's going on. And when an error occurs, the compiler spits out a message having the length of spam email.

A Simple Idea

My proposal is very simple. It's based on the idea that the programmer should see in the error messages the same names of types that he/she uses. It's obvious that no one is likely to use a 100-character name as a type. Anyone would replace such a name with a type definition defining a shorter name. This means that the compiler should collect, and use in diagnostic messages, information about typedefs and default template arguments. The change is likely to be noninvasive in the architecture of many compilers. Some additional information must be stored in symbol tables, to be used only in case of an error.

That being said, the proposal boils down to the following few guidelines:

1. An error message involving an expression should contain the type definition (typedef), if any, that was used by the programmer to originally express the type of that expression. The typedef will be prefixed with scope information appropriately. For instance, if the typedef occurs in a namespace/class/function, that scope name must appear in the diagnostic message as well.

For instance, the code fragment:

// ... includes omitted
using namespace std;
typedef map<string, double> TDblMap;
typedef vector<TDblMap> TDblMatrix;
void f(const TDblMatrix &M)
{
    M[0] = M[1];
}

would yield the following error message:

** binary '=' : no operator defined which takes a left-hand operand of type 'const TDblMap' (or there is no acceptable conversion)

instead of:

** binary '=' : no operator defined which takes a left-hand operand of type 'const class std::map<class std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>>, double, struct std::less<class std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>>>, class std::allocator<double>>' (or there is no acceptable conversion)

Rationale: Typedefs are introduced by humans for humans. Their purpose is exactly to simplify the expression of complex types. Error messages are intended for humans as well, so basing them upon typedefs comes naturally.

2. Template parameters that resulted from default parameters in template classes and functions should not be specified in error messages. Put another way, if you always use the default allocator in STL containers, you'll never see it in any error message.

For instance, the code fragment:

using namespace std;
void
f(const map<string, double> &Salary)
{
    Salary["John Doe"] = 100000;
}

would yield the following error message:

** binary '[' : no operator defined which takes a left-hand operand of type 'const class std::map<class std::string, double>' (or there is no acceptable conversion)

instead of:

** binary '[' : no operator defined which takes a left-hand operand of type 'const class std::map<class std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>>, double, struct std::less<class std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>>>, class std::allocator<double>>' (or there is no acceptable conversion)

Rationale: An error message that contains the default template arguments is redundant. The presence of the defaults does not help much in understanding the meaning and cause of the error. On the contrary, defaults are most often intended to protect non-expert programmers from the complexity of a highly customizable library.

3. Errors that appear in template classes, functions, and methods should hint also to the line that caused the template to be instantiated. There will be two or more error messages generated: one at the point that caused the instantiation, and the next one(s) at the point(s) that caused the concrete error(s).

I have already seen this behavior implemented in the free, brand-new egcs C++ compiler (http://egcs.cygnus.com/) and indeed find it very useful.

For instance:

using namespace std;
void f(const vector<double> &DblVec)
{
    sort(DblVec.begin(),
         DblVec.end());
}

would yield:

** main.cpp(15): instantiating std::sort(vector<double>::const_iterator, vector<double>::const_iterator): 2 errors (see below)

** algorithm(579): l-value specifies const object

** algorithm(580): l-value specifies const object

instead of only the last two messages.

Rationale: This is of great help to terrorized newcomers, who find errors in library code and think they have an installation problem. Even after I got accustomed to STL I had a lot of trouble in removing such obscure errors from hundred-lines modules when I ported code from one compiler to another. Basically I had to do a binary search by commenting code in and out :o). So I think this rule is indispensable in helping to spot problems. For instance, in the example above, the presence of const_iterator instead of iterator in the error message is an excellent hint even for a beginner.

4. Non-type template arguments introduced as symbolic constants by the programmer should be kept in symbolic form instead of being translated to numeric/address form.

Rationale: More often than not, symbols are better than magic numbers.

Caveat

I think we could imagine cases where such error messages could be misleading. For instance, consider this situation:

typedef long HRESULT;
HRESULT g();
...
std::string f()
{
    return g();
}

The error message would be:

** cannot convert from HRESULT to std::string

and the user might ask, what class is HRESULT after all? The compilers might add a diagnostic mode where the full-blown names are listed along with the pretty messages.

Request for Comments

Those who've read Barton and Nackman's article [1] will immediately see what a huge difference there simple improvements could make for people whose expertise in C++ is not great. I am eager to hear your opinion about this proposal, and hopefully refinements and improvements on these simple rules as well. I would also be grateful to CUJ if they could save one square inch in "We Have Mail" in the next few issues for a forum on this subject.

Reference

John J. Barton and Lee R. Nackman, "Dimensional analysis," C++ Report, January 1995.


Steve Clamage, Sun Microsystems, Inc.

I'll organize my comments according to the sections in the proposal. I can present my comments from the viewpoint of a C++ compiler user as well as a C++ compiler implementer.

The Problem — Long Names in Error Messages

I couldn't agree more. As we implemented and tested the Standard C++ library, we of course encountered many errors. Because the standard library is so heavily templatized, names in error messages were excruciatingly long, making it very difficult even to read the error messages. Believe me, no one is more aware of the problem than a C++ implementer!

1. Use the typedefs in error messages that the programmer used.

I agree that this would typically make the messages more readable. It's more work for the compiler — it has to keep around not just the types of expression elements, but the way that type was expressed. In the case of our compiler, it would mean a restructuring of the way we keep program information and generate names for error messages. It's a significant amount of work, although it's worth it.

I'm also concerned that the abbreviated names would sometimes be more confusing than the fully expanded names. For example, the programmer might believe that the abbreviated name referred to one type when it really referred to another type with the same name in a different scope. (The lookup rules for names in templates are very complicated, and sometimes the result you get is not what you thought you were going to get.)

I think the fully expanded name must also be available somehow. With interactive error reporting, you would click on an "expand name" button to see it. With traditional reporting, you'd either get both versions of the error message, or recompile with different options to get full names. I don't know the right answer, or whether there is a "right" answer.

2. Elide default parameters from types when the defaults are used.

All my comments for item 1 apply here.

3. Errors in instantiations should indicate the line in the template and the line where the instantiation occurred.

I agree completely. In fact, the Sun C++ compilers have done exactly that for several years. When sequential (e.g., nested) instantiations are involved, you get a "walk-back" showing the sequence.

4. Express non-type template arguments in their symbolic form.

This sounds reasonable, especially when an address is involved. Sun C++ does that, in fact. That is, if a class takes a function as a template parameter, the class type is expressed in error messages using the function name, not its address. (The address isn't known at compile time anyway — the function doesn't get an address until link time, assuming the program links.)

Now consider numeric arguments that are computed:

MyClass<(sizeof(A)
    + sizeof(B) + 7) / 8>

Should an error message show the textual representation of the expression? We probably want to know the actual value used as well. If I'm trying to figure out why something is out of range, I'd like to know what the compiler thinks the range is.

In the C++ compiler group at Sun Microsystems we have discussed these issues from time to time, without coming to any firm conclusions. I'm glad to see a public discussion, and I'd hope to be able to use the results of any consensus in a future compiler release.


John Spicer, Edison Design Group

The author of the proposal makes some reasonable suggestions about how the compiler's error messages could be improved. The EDG (Edison Design Group) front end produces messages very similar to the ones suggested in the first and third examples in the proposal. Even so, we continue to receive requests from our customers to produce better template diagnostics.

In the EDG front end, we try to use names from the original source program as much as possible (as is suggested in the first proposal), but this sometimes creates surprises for users. I've modified the first example so that the error comes up as the result of using three different names for the same type. Which one should be used in the error message?

typedef map<string, double> TdblMap;
typedef vector<TdblMap> TDblMatrix;
     
typedef map<string, double> my_TdblMap;
typedef vector<my_TdblMap> my_TDblMatrix;
     
template <class T> inline void g(T& t)
    {
    t[0] = t[1];
    }

void f1(const TDblMatrix &M)
    { g(M); }
void f2(const my_TDblMatrix &M)
    { g(M); }
void f3
(const vector<map<string, double> > &M)
    { g(M); }

We end up using TDblMap in the message, but some users would prefer to see map<string, double> if, for example, the name TDblMap happened to be used deep inside some header file they used and they had no idea what the underlying type was. In other words, sometimes "obvious" improvements don't always work out as well as expected.

The third part of the proposal suggests that the point of reference that caused the instantiation should be provided for an error that occurs within a template. The example in the proposal shows only the outermost point of reference and the innermost template. What is really needed is a full traceback of the templates whose instantiation ultimately results in the error.

For example:

d:\progra~1\devstudio\vc\include\algorithm", line 579: error: expression must be a modifiable lvalue
*_L = *_M;
^

detected during:

instantiation of "void std::_Unguarded_insert(_BI, _Ty) [with _BI=const double *, _Ty=double]" at line 572

instantiation of "void std::_Insertion_sort_1(_BI, _BI, _Ty * [with _BI=const double *, _Ty=double]" at line 565

instantiation of "void std::_Insertion_sort(_RI, _RI) [with _RI=const double *]" at line 538

instantiation of "void std::_Sort_0(_RI, _RI, _Ty *) [with _RI=const double *, _Ty=double]" at line 534

instantiation of "void std::sort(_RI, _RI) [with _RI=const double *]" at line 8 of "main.cpp"

This example shows something else that we've done to make the messages simpler to interpret. Template function names are specified by providing the function template signature and the template argument list rather than giving the function signature of the specialization. For example, the EDG front end says:

std::sort(_RI, _RI) [with _RI=const double *]

instead of saying:

std::sort(const double *, const double *)

You could argue about which is better for this particular example, but our format is particularly helpful when there are several templates that could potentially generate the same specialization (i.e., when making use of partial ordering of function templates) or when the template argument value is a more complex type.

While suggestions like these should be explored to make template messages clearer, some users have suggested to us that a higher-level mechanism is needed to make libraries like STL more usable for non-experts. In the example above, wouldn't it be nice to get a message like

"main.cpp", line 8: the argument to std::sort must be a mutable iterator

Such a mechanism would work by letting template writers describe the constraints that template arguments must meet for the template to be successfully instantiated. Of course, the language has no means to specify such constraints, so a language extension or an extra-lingual mechanism (such as pragmas) would be required.

I suspect that most compiler vendors have had their hands full just completing all of the features that are in the new standard. Once this is done we can turn our attention to making those features more usable. In the meantime, users should contact their compiler vendors when they get messages that are difficult to use. Concrete examples of the problems that come up in real world code are of tremendous help to compiler vendors that want to improve their diagnostic messages.


Jonathan Caves, Microsoft Corporation

Why do compilers give such terrible error messages? It is a sad but true fact that diagnostics are one of the most overlooked aspects of compiler development. While the standard of compiler error messages has slowly improved over the years (at least we have got away from the overused "syntax error") there is still much room for improvement. I can come up with many reasons for bad compiler diagnostics, but I will limit myself to what I see are the three major ones.

The first reason for the poor state of compiler diagnostics is historic. All software used to have to operate in highly constrained memory. Compilers, especially C and C++ compilers, tend to be pretty memory hungry beasts. Therefore, in order to get the performance users expect, they tend to be highly optimized. One sure way to get better performance has always been to try to reduce the working-set as much as possible. For a compiler developer, this meant trying to reduce as much as possible the amount of information the compiler needs to hold onto during a compile.

So consider the case of typedefs. The compiler is only interested in the underlying type, not what the user has chosen to name the type. Thus the compiler immediately forgets all about the typedef and only remembers the underlying type. If later the compiler is required to give a diagnostic that references the type, it will emit the bare type instead of the typedef.

A second reason for bad diagnostics is the compiler developers themselves. In general, compiler developers are extremely knowledgeable about the specific language they are developing a compiler for. Unfortunately these compiler developers are also the authors of the error messages. When faced with a situation that requires a new error message, they tend to reach for the language specification or standard. From this they produce an error message that succinctly describes the error.

Unfortunately these error messages are usually incomprehensible to anyone but other compiler developers or language lawyers. Do we really expect the average user to know the difference between an l-value and an r-value? Does anyone other than a compiler developer really need to know about a declaration-specifier? Exactly what is the difference between a type-modifier and a type-qualifier? Who can tell the difference between an explicit specialization of a class template and an explicit instantiation of a class template?

The last reason for bad diagnostics is due to the way software is developed. Whenever a new product, or a revision of an existing product, is planned the developers get together with Marketing, Quality-Assurance, and Program Management to decide on the list of features that will make up the next release. Usually this list will include must-have features, should-have features, and would-like-to-have features. Unfortunately, better error messages is rarely a must-have feature. So as the development cycle progresses and features slip and priorities change, the would-like-to-have features drop off the list. Somehow, when faced with a choice between better COM support, or enhanced template support, or better diagnostics, better diagnostics never wins.

We all have trade-offs to make between user requests, tactical and strategic feature enhancements, and bug fixes. User requests, like this one for better diagnostics, have been improved over time. However, we must always make the tradeoff between working-set size, benefit of the improvement, and the need to implement other important features.

As for the specific points raised by Alexandrescu, these features would all be relatively easy to implement within a modern compiler — especially as memory is no longer the huge constraint it used to be. But all these changes assume that neither the compiler developer nor the end user mind the increase in working-set that would happen as a consequence of the compiler holding on to much more information about the program.

The compiler could keep track of use of individual typedefs instead of just dealing with the underlying type. I think that if we made only this change, it would produce much clearer error messages.

Keeping track of which template arguments the user specified and which were defaulted is again a relatively easy addition to make to a compiler. One problem with this approach — especially when applied to STL — is that it will only work on top-level elements of STL. The lower-level elements all tend to have their template arguments fully specified. Though the compiler could solve this problem it would just take more time. (And space!)

The third proposal is one that we have considered for some time, and Visual C++ does implement it to a certain extent. As the author has stated, with templates the error itself is sometimes irrelevant. What is much more useful is the path the compiler took to get to the situation that caused the error. Which template instantiations caused the compiler to need to instantiate the current class?

The fourth proposal, like all the others, is possible within a compiler. (One of my manager's favorite expressions, when faced with someone telling her that something wasn't possible, was "It's a compiler — anything is possible.") But as I've stated before, there is the eternal tradeoff between space and speed. There is no reason that the compiler couldn't generate an in-memory representation of a whole program. It could then generate very clear error messages detailing not only exactly what caused the problem but also suggesting possible fixes. Such a compiler is possible, but I doubt if many of today's users would find its performance satisfactory, even on the memory-rich machines we are all now accustomed to.

Andrei Alexandrescu is a developer with Micro Modeling Associates, Inc.'s New York Component Solutions Group. He is responsible for application development using Visual C++, ActiveX technology, Visual SourceSafe, ODBC, and SQL Server. He may be reached at alexandrescu@micromodeling.com.