Extend each entry in the call stack backtrace (hereafter called
stacktrace) returned from the erlang:get_stacktrace/0 BIF and from the
catch operator with filenames and line number information.
Currently a stack trace returned from erlang:get_stacktrace/0
(and the catch operator) is a list of three-tuples, where
each tuple looks like:
{Module,Function,Arity}
(In some cases, the third element may be a list of arguments instead of the function arity.)
We propose to change each tuple to:
{Module,Function,Arity,LocationInfo}
LocationInfo is a property list (a list of two-tuples) that
contains filename and line number information. If there is
line number information available, the list will look like:
[{file,FilenameString},{line,LineNumber}]
The list should be accessed using proplists:get_value/3 or
lists:keyfind/3, not by direct matching, since a future release
may add more items to the list or change the order.
The filename is usually the same as the module with the extension “.erl” added, but if function definitions have been placed in a header file, the filename will be the name of the header file. The filename will also be different if the Erlang source file has been generated by a code generator such as yecc.
The line number will never be zero; instead LocationInfo will be
set to an empty list.
The list will be empty if there is no location information available. Here are some reasons that location information may be missing:
The module has been compiled with an older BEAM compiler that does not support generation of line number information.
The module was created by calling compile:forms/1,2 with forms
that did not contain non-zero line numbers and/or filenames.
A parse transform created abstract forms having the line number zero.
The module was created using an alternate compiler that did not provide filenames and/or (non-zero) line numbers.
The line number information may have been stripped from the BEAM file.
The exception occurs in a BIF (implemented in C in the run-time system).
This EEP does not specify exactly how line number information should be implemented, but it does impose some requirements on the implementation:
The presence of line number information should have (virtually) no impact on the execution time for a program if no exceptions occur. In practice, that means that an implementation is not allowed to add extra instructions or BIF calls that will be executed when no exception occurs.
Line number information should not be dependent on debug information being present in the BEAM file.
Line number information should be included by default in BEAM files. (There could be options to turn off the inclusion of line number information.)
Loading line number information should be the default. There may be an option to turn off loading of line number information in order to save memory.
In the examples, we will use the following module:
-module(example).
-export([m/1]).
-include("header.hrl").
m(L) ->
{ok,lists:map(fun f/1, L)}. %Line 6
and the header file header.hrl:
f(X) ->
abs(X) + 1. %Line 2
Using R14B01 to call our example module, we get the following result:
1> example:m([-1,0,1,2]).
{ok,[2,1,2,3]}
2> example:m([-1,0,1,2,not_a_number]).
** exception error: bad argument
in function abs/1
called as abs(not_a_number)
in call from example:f/1
in call from lists:map/2
in call from lists:map/2
in call from example:m/1
3> catch example:m([-1,0,1,2,not_a_number]).
{'EXIT',{badarg,[{erlang,abs,[not_a_number]},
{example,f,1},
{lists,map,2},
{lists,map,2},
{example,m,1},
{erl_eval,do_apply,5},
{erl_eval,expr,5},
{shell,exprs,7}]}}
In a system with line number information enabled, we get:
1> example:m([-1,0,1,2]).
{ok,[2,1,2,3]}
2> example:m([-1,0,1,2,not_a_number]).
** exception error: bad argument
in function abs/1
called as abs(not_a_number)
in call from example:f/1 (header.hrl, line 2)
in call from lists:map/2 (lists.erl, line 948)
in call from lists:map/2 (lists.erl, line 948)
in call from example:m/1 (example.erl, line 6)
3> catch example:m([-1,0,1,2,not_a_number]).
{'EXIT',{badarg,[{erlang,abs,[not_a_number],[]},
{example,f,1,[{file,"header.hrl"},{line,2}]},
{lists,map,2,[{file,"lists.erl"},{line,948}]},
{lists,map,2,[{file,"lists.erl"},{line,948}]},
{example,m,1,[{file,"example.erl"},{line,6}]},
{erl_eval,do_apply,5,[{file,"erl_eval.erl"},{line,482}]},
{erl_eval,expr,5,[{file,"erl_eval.erl"},{line,276}]},
{shell,exprs,7,[{file,"shell.erl"},{line,666}]}]}}
If we compile the example module using the BEAM compiler in R14B01,
there will not be any line number information for that module:
1> example:m([-1,0,1,2,not_a_number]).
** exception error: bad argument
in function abs/1
called as abs(not_a_number)
in call from example:f/1
in call from lists:map/2 (lists.erl, line 948)
in call from lists:map/2 (lists.erl, line 948)
in call from example:m/1
2> catch example:m([-1,0,1,2,not_a_number]).
{'EXIT',{badarg,[{erlang,abs,[not_a_number],[]},
{example,f,1,[]},
{lists,map,2,[{file,"lists.erl"},{line,948}]},
{lists,map,2,[{file,"lists.erl"},{line,948}]},
{example,m,1,[]},
{erl_eval,do_apply,5,[{file,"erl_eval.erl"},{line,482}]},
{erl_eval,expr,5,[{file,"erl_eval.erl"},{line,276}]},
{shell,exprs,7,[{file,"shell.erl"},{line,666}]}]}}
The lack of line number information in exceptions is a major stumbling block for many beginners, and is a time waster for experienced Erlang programmers.
An often repeated piece of advice to mitigate the lack of line number
information is to write smaller functions. To some extent, that is
good advice, but some functions are most naturally written as a single
function with many clauses. One example is the handle_call/3
callback for a gen_server process. Another example is test suites.
In a typical test suite, every line tests a condition and can
potentially fail. It is not practical to put every line that may fail
in a separate function.
Test suites based on common_test are automatically run through a
parse transform that provides line number information when an
exception occurs. The parse transform inserts before every line code
that saves the current function name and line number in the process
dictionary. When an exception occurs, the line number can be
retrieved and presented.
One problem with this approach is that the test suite will run slower, which can cause test cases to fail if timeouts expire in the system being tested. Another problem is that by default the parse transform is only run on the test modules themselves, and therefore exceptions that occur in other parts of the code (support libraries for testing or the product itself) does not have any line information.
We have chosen to let erlang:get_stacktrace/0 and the catch
operator return stacktraces with filename and line number information
(instead of introducing a new function called, for example,
erlang:get_full_stacktrace/3). That means that code that simply
passes on the stacktrace (to erlang:raise/3) does not need to be
updated. For example, the following code that catches an exception,
logs it, and pass it on does not need to be updated:
try
some_call_that_may_fail()
catch
Class:Reason ->
Stk = erlang:get_stacktrace(),
log(Class, Reason, Stk),
erlang:raise(Class, Reason, Stk)
end
One the other hand, that means that code that assumes that the stacktrace only may contain three-tuples will no longer work and needs to be updated.
There are several reasons for the requirement that the line number information should be loaded by default (rather than ordered by giving an option).
In real systems, code size is usually not an issue since it is overshadowed by the memory used for process heaps, off-heap binaries, and ETS table. Therefore, the 10 percent increase of the code size (as measured in the reference implementation) is not an issue for most users, but the benefit of having line number information is potentially huge.
Newcomers to Erlang have the most need for line number information and they should get it without giving any special option. If an option is needed, questions to the mailing lists about how to find from which source line an exception was caused will continue to waste time.
If an option must be given, even developers that know about it may forget to give it and might therefore end up having to investigate an exception without line number information. (Which may waste a lot of time if the problem is not easily reproduce-able.)
Therefore it is better that the developers that cannot afford any increase in the size of the loaded code are the ones that must give an option to turn off loading of line number information.
Applications that examine the stacktrace and assume that it contains
three-tuples must be updated. The erlang:raise/3 BIF still accepts
three-tuples (it will translate those to four tuples with an empty
list in the fourth element); thus it is not mandatory to update calls
to erlang:raise/3.
The reference implementation can be fetched from Github like this:
git fetch git://github.com/bjorng/otp.git bjorn/line-numbers-in-exceptions
Here is an overview of the implementation:
The BEAM compiler inserts a line instruction before every construct
that may generate an exception and before every call that will be
included in the stacktrace. (Local tail-recursive calls need no line
instruction, but external tail-recursive calls need a line
instruction because they may be calls to BIFs.)
The line instruction has a single operand, an index into a line
number table. The line number table is stored in the “Line” chunk in
the BEAM file. The “Line” chunk and line instructions increase the
file size of BEAM files by about five percent.
The loader will remove the line instructions from the code that will
be executed, but will remember their location and create a table
sorted in address order mapping from program counter to line number
information. When a stacktrace needs to be built, the run-time system
will do a binary search for the program counter of exception-causing
instruction and each continuation pointer.
For the benefit of embedded system that run in a very constrained
memory space, the run-time system can be started with the ‘+L’ option
to disable loading of the line number information. The code will
still be about one percent larger than code compiled without line
number information, because the compiler was unable to do code sharing
optimizations on instructions that cause exceptions (such as the
badmatch instruction).
In the current implementation, the line number information increases the size of the loaded code by roughly ten percent.
This document has been placed in the public domain.