The Necessity of Exceptions

Exceptions often get a bad rap. They are called ineffective, inefficient, and hard to work with. In a way such accusations aren’t unfair, as most languages have totally screwed up their implementation. Hatred of exceptions is mainly a backlash against poor syntax, bad compiler implementation, and unmanageable exception hierarchies. Despite these stumbling blocks, exceptions are still far better than the alternative: no exceptions.

By Example

Consider the following pseudo-code. The function “sum_file” sums up the integers on each line in the file.

int sum_file( string name )
{
	File in = open( name );

	int sum = 0;
	while( !end_of_file(in) )
	{
		sum += parse_int( get_line(in) );
	}

	return sum;
}

In this form the intent and flow of the function is very clear. You don’t have to study it for long to see what is happening. In particular there is no error handling cluttering up the function. We have decided that if an exception occurs the caller will have to deal with it. This is reasonable since we would have no valid return value if that were to happen.

What would this look like if we didn’t have exceptions? What if we had to handle errors individually and find a way to propagate them to the caller. This code is below. Here we’ll even assume that we have good language support for multiple return values: a return tuple where the first value is the real result, and the second the error value.

int,error_code sum_file( string name )
{
	error_code err;
	File in;
	in,err = open( name );
	if( err )
		return 0,err;

	int sum = 0;
	while( true )
	{
		bool eof;
		eof,err = end_of_file(in);
		if( err )
			return 0,err;
		if( eof )
			break;

		string line;
		line,err = get_line(in);
		if( err )
			return 0,err;

		int value;
		value,err = parse_int(line);
		if( err )
			return 0,err;

		sum += value;
	}

	return (sum, okay);
}

Yuck! The purpose of our code is entirely lost in error handling. It is now very difficult to see what the main purpose of the code is. Even worse, we’re forced to return a result even if one isn’t available, so we return 0. If the caller forgets to check the error code they will simply use 0, which is most likely not what is intended. It is very easy to ignore return codes.

You may protest that such a level of error handling is not required. In the wild you’ll indeed find functions like the above which lack several of the checks. But you’d be wrong and those functions would be broken. Every check for an error in the above is necessary to catch some kind of error condition. Consider even “end_of_file”, though rare it can actually fail. What if the USB stick was removed in the middle of your function and the file just doesn’t exist anymore? If the function simply returns “true” instead of an error, you won’t sum up all the values and thus you’ll have a defect. Even in the above we may be missing a check on the “+=” operator. We’re just assuming integer addition can’t fail, but in some languages it can, and instead of a raw integer we might be using a BigDecimal type instead.

Contextual Information

In the case of error we also have very little ability to report contextual information to the caller. All we can report is a simple error code. Anybody who has done work with C system functions knows how limiting this can be. I even had a case recently with “dlopen” simply returning a “file not found” code. That wasn’t actually the problem, but it didn’t really have a way to report anything else. In the above code possibly the caller will just further return the same error code, and every step up we take we lose contextual information about what actually happened.

With an exception the lower function in the chain has the ability to better explain what happened. For example, if the USB stick was removed, the thrown exception can actually contain information indicating that error and possibly what mount point it had, or the name of the file. While a single bit of extra information isn’t a lot, it can often mean a great deal while debugging.

Ideally each function in the chain could add a little bit of information to provide a full context. This would be a tagged exception system. Unfortunately I don’t know of any natural support of such a system. I only know boost exceptions in C++ which allows you to use tagged exceptions — truly an excellent approach once you understand it. Not having this system greatly reduces the ability of an exception to report context. More on this in a future article.

The Way Forward

This article is not intended to be a thorough exploration of exceptions. The examples are simply here to show that exceptions allow much cleaner code than a lack of exceptions. Code which cannot rely on exceptions ends up bloated and obscuring all intent with an excessive amount of error handling. Clearly, exceptions are a very powerful tool and should be considered a standard feature in any new language.

Don’t misunderstand this to mean that exceptions should always be used. There are certainly domains where an exception flow is not desired. But those cases are not the norm. Exactly when you shouldn’t use exceptions is a difficult subject; too much to cover here. And perhaps these cases only exist because current languages have implemented exceptions very poorly.

Yes, you read that correctly. Despite my assertion that languages need exceptions I maintain that most languages have gotten them terribly wrong. We should still use them, but we should also be looking at a way to improve them. Boost’s C++ tagged exceptions looks to be a very promising approach. In time I’ll look further at what went wrong, and how to improve it.

Followup

There appears to be some debate that I am simply not doing any error handling at all in the first version (the one where I claim exceptions do the work). That is part of the point I am making, that function has no way to deal with an exception therefore just passes it up to the caller.

Obviously somebody up the chain will have to deal with it. Here is a possible caller who wishes to deal with it directly, just to show the above functions are in fact complete.

//with exceptions
try {
	int sum = sum_file( a_file );
	printf( "The Sum is %d", sum );
} catch( Exception ex ) {
	printf( "Sorry, it failed: %s", ex.to_string() );
}

//with error codes
int sum, err;
sum, err = sum_file( a_file );
if( !err )
	printf( "The Sum is %d", sum );
else
	printf( "Sorry, it failed: %s", code_to_text( err ) );

42 Comments

  1. Troels Henriksen

    The first point of your critique of exception-free systems, that they add boilerplate and easily miss error cases, could be solved by running the code in an error handling monad. The Haskell monad Maybe can capture only binary success/failure, but the Either monad can capture either success or a failure characterised by some well-typed value. The benefit compared to most current exception systems is that the failure modes (the type of the monad) is expressed in the type system, and you will receive a type error if you do not handle it properly. For example, here is your example rewritten in pseudo-Haskell (the loop changed to a recursive function):

    sum_file :: String -> Either SomeErrorType Int
    sum_file name = do
    in <- open(name)
    let sum_file' sum =
    if end_of_file in then return sum else do
    line <- get_line(in)
    value <- parse_int(line)
    sum_file' (sum+value)
    sum_file' 0

    The trick behind the do-notation (which is syntactic sugar for monadic binding) is that if any of the calls to the right-hand-side of an arrow ("<-") fail, the entire do-block will fail. Of course, this is very similar to a standard exception system, except I think it makes it much more clear when something might fail, and how.

    It probably has too much overhead for truly exceptional situations, however: while you can run out of memory while adding two integers, it would be very cumbersome if every addition had to be executed in the monad.

    1. mortoray

      I’m looking more into Haskell’s system now. I thinking somewhere between the combination of how errors are handled in all languages there must be some kind of optimal approach.

      I do intend on looking further at typed/checked exceptions and I’ll try to keep Haskell in mind.

    2. aaronla

      The Either monad (like Maybe, except that None can carry an error object with is) is isomorphic to exception unwind.

      There’s not really a lot of mystery to it either — exception handling makes the typical case (error propagation) implicit instead of explicit, and the atypical case (ignoring an error) explicit instead of implicit. The Either and Maybe monads do the same within ‘do’ syntax — sequencing is mapped to propagation, not dropping.

  2. snake5

    The no-exception example of yours is seriously wrong. No sane person would design an API like that – it’s possible to store error in the file object and fail silently. In which case EOF would be reached prematurely and all errors could be parsed at the end of this function (or thrown when the object is destructed to notify the programmer of their existence) without requiring the cost or complexity of exceptions.

    Also if you would use exceptions there, all of this mess would be sent to the higher level code (which you conveniently didn’t show anywhere), where you’d have to check if exceptions of certain kind have occured and react accordingly. That’s not a good way to write readable high-level code.

    Therefore my conclusion goes like this: exceptions are only useful to patch up awful API designs.

  3. mortoray

    In the first example this code has no way to deal with the exception other than to send it up to a higher level. This is a key part to this example. There is absolutely no way for this function, as shown, to deal with the error other than generate an error on its own. And yes, defintiely a higher level function will have to handle it. In a well designed system there should be very few places that actually handle exceptions. Code which has a lot of exception handling is also designed wrong — and I’ll get into that in another article.

    As to accumulating the error in the file. Yes, that is an option in some cases, but not always. I find it needlessly complicates the API and sets up a series of unusual expectations. While reading a string from the file, if you ran out of memory, why would the file be marked as an error? Even if we agree that is okay, you still need error handling at the parsing of the integer, and at the end of function, which is still more than should be needed for the function.

    1. snake5

      Running out of memory is not an exception, it’s a fatal error. In case memory can be freed up somehow, it must be done in the allocation function, not in some exception handler, to avoid doing low-level work in high-level code.

      Parsing invalid input data is almost covered by having the file error that states that all data you have received from the moment the error was set is wrong.

      In case the data is wrong, not the file, parsing that one error should not trigger a catastrophe.
      In this example, the preferred action would probably be to simply avoid adding the number or, in other words, adding a zero – which is what the atoi function conveniently returns in case of bad input data.

    2. g2-bbf876e123027b9573d3a25a8a63bf6b

      > In this example, the preferred action would probably be to simply avoid adding
      > the number or, in other words, adding a zero – which is what the atoi function
      > conveniently returns in case of bad input data.

      And silently ignore errors !? That’s awesome design.

      Maybe integer parsing should have better API, but silently returning 0 is no better that throwing exception on bad input.

      The 0, and other “ignore bad input” are all hacks to improve maybe interoperability of systems but they are hacks.
      It’s not default and it can’t be default.

      What if you design a product function? Is the zero-returning atoi still so good option ?

    3. snake5

      g2-bbf876e123027b9573d3a25a8a63bf6b:

      Error handling heavily depends on the situation. In this case, returning 0 on error is a very good option (better than explicit error checking of any kind). In others, it may not be so. Is that so surprising?

      For web development, values of different types are returned on error – for example, false / null in PHP. It is actually an extremely efficient way to handle errors because if you don’t want to do it, you get an automatic conversion to default value and if you do, you do a strict equality test on output and handle everything accordingly. No tuples, no exceptions and no restrictions.

      More often than not, the reality is that most errors of such a small scale don’t need to be checked. But only if the goal is to write code that can be used as quickly as possible, not some abstract notion of ‘perfect code’…

    4. Matthieu M.

      This really is a C-like mentality.

      No, silently dropping on the floor the fact that the parse failed is not a good feature for a language. You should give the caller the option to either or care not, and perhaps by default ignore it unless an option is passed, but when *I* parse a file, I expect to be notified of errors so that I can look them up and understand whether my understanding of the format I was supposed to parse is wrong or if I somehow got junk.

      But then, I build reliable software, not webapps.

    5. Andrzej Krzemieński

      I disagree with the statement that “running out of memory is a fatal error”. It appears much more convenient (I say “appears” because I never experienced this problem in practice) to throw an exception in case of memory outage. In a design where exceptions are only handled sparingly in higher levels of the application, the stack unwinding mechanism (I am thinking of C++) will call destructors of automatic objects. The destructors in turn will likely free the heap memory (because this is the most common resource to manage in destructors); and as the stack unwinds, the amount of available memory increases. If you just catch this exception in the top layer of the application, it is likely (although not certain) that the program has already reclaimed sufficient memory.
      Then, if you want to treat out-of-memory situation as fatal, it is also convenient with exceptions: just do not catch this exception – let it stop the application.

  4. minhen

    If you design exceptions in a way that they contain unique information for each exception, then you end up having to deal with that uniqueness in higher level functions. This is almost as bad as expecting callers to handle different types of exceptions excplicitly like in Java. In general the design of programming languages should support and simplify the common usage, which is using frameworks and not writing them. Therefore burden callers with enriched and therefore more error handling is exactly the wrong way to go (though it follows the traditional way of thinking when it comes to exceptions).

    A better yet old school approach is the one of Objective-C. Exceptions are only thrown in fatal situations, so in general there is no need to even bother to handle exceptions as a caller. Normal errors are signaled by returning NO or nil and by optionally providing a NSError object, which contains error codes, error domains, some additional but not arbitrary context and even localized error descriptions. In addition calling methods on nil simply does nothing and causes no harm. This gives some degree of freedom in error handling. And NSError provides all relevant information in a consistent and predefined manner, which allows to handle all errors the same way, which in turn simplifies the code. Less code, less bugs. It’s by no means fancy or interesting but the code gets more readable and more maintainable. And in the end that’s what matters.

    1. mortoray

      I absolutely agree that a rich exception hierarchy (ala Java) is junk. In projects where I use exceptions the hiearchy is very flat (3 classes total, one base, 2 derived). I am strongly in favour of a tagged exception hierarchy: the details are there, but you only use them if you want, otherwise you can deal with very simple exceptions.

      I intend on going into this more in another article.

    2. Matthieu M.

      I also tend to use flat hierarchies. Either an exception occurred or it did not.

      I find the binary hierarchy you mentionned excellent for online transaction processing: either the error is unrecoverable (process corrupted) or just the current transaction failed. Further details (stack trace at least, notes and details as in tagged exceptions) can be embedded within and need not clutter the “type hierarchy”.

  5. Alex Scheel Meyer

    For some very related discussion on how to best do error handling in your example, have a look at this video : http://youtu.be/VeAdryYZ7ak?t=42m25s

    The essential notion is that it is not enough to consider failing silently (as in your first example) or catastrophically (as in using exceptions, tagged or not), you also need to consider having a mechanism for dealing with failure which is not determined by the failing code but at some higher point in the call hierarchy.

    Specifically in your example one might want to ignore the lines in the file that are not parseable (like empty lines placed for readability) and only sum on the lines that actually contain integers. Such a policy needs to be dictated from outside of the function.

    1. mortoray

      I am not advocating failing silently. In the first example I fully expect someething at a higher level to deal with the problem. At the level of the sum function itself however there is no appropriate course of action, and thus deferraly to the caller is appropriate.

      I will also agree with what I believe is your notion that you should try and provide an API that simply doesn’t have so many error conditions.

    2. Matthieu M.

      I really like the idea of passing the policies (Lisp did it in a quirky way…) however the problem is that at some point you have to take a decision on granularity.

      This simple function may fail in roughly 3 different ways:
      – absent file
      – file reading issue
      – wrong format

      It could accept up to 3 different policies!

      Now, imagine the caller does not care much, and calls another 3 or 4 functions which accepts 3 policies… should it ask from its own caller the whole 15 policies!

      At some point, aggregation is necessary to preserve sanity. This is what typical exception hierarchies try to get to in general.

  6. cschneid

    You’re unnecessarily obfuscating the version without exception.
    One you also write it similar to this (leaving out the variable declarations for brevity).

    in, err = open(name);

    while (!err && !end_of_file(in))
    {
    line, err = get_line(in);
    if (!err)
    value, err = parse_int(line);
    if (!err)
    sum += value;
    }

    return (sum, err);

    1. mortoray

      I’ll admit there are always opportunities to make it shorter. The problem is how to make it more compact varies with every situation, thus I consider my example as a valid example of the general form one must use.

  7. Henry Eshbaugh

    In regards to the pseudocode you used as an example:

    True enough, the second example is a lot more convoluted than the first. But, at the same time, neither of them actually *do* anything. The call is an intermediary; you’re relying upon open(), end_of_file(), get_line(), and parse_int() to throw exceptions. You don’t introduce anything because nothing could go wrong with your code itself, only what you call.

    Move onto real-world code, though, and you have a problem. Say you’re writing a function to parse integers from some memory address given by the caller. Here, you’re doing something, and here, something can go wrong. The address might be NULL. The address might not be to data you want. The length of the buffer could be invalid, and a whole slew of other problems. All of these conditions need to be checked for, and guess what? it’s you’re responsibility to check it. That means lots of messy if statements. The only difference is, instead of using return error_code, you’re using throw(error_code).

    So, half the basis of your argument – that not using exceptions is ugly – is just plain wrong.

    Granted, error handling is tricky without exceptions. But, let’s face it – exceptions are inefficient (as you said yourself), and so the only advantage, that it saves developer time, comes at the expense of user time. Remember, there’s only one of you, and hundreds (if not thousands or more) consumers, and so one little thing to save you time is going to add up to a whole lot more time wasted because you chose the easy way.

    1. mortoray

      Defintiely some code will need more error handling. My conjecture however is that the vast majority of functions are these “middle” functions that don’t do much error handling on their own, either none, as in the example, or only check a few cases. The point is that error handling should only appear in places where it is strictly necessary, not in every function along the way.

    2. Allan Wegan

      > Remember, there’s only one of you, and hundreds (if not thousands or more) consumers, and so one little thing to save you time is going to add up to a whole lot more time wasted because you chose the easy way.

      That there is only one of him for every huge bunch of users – that is exacly the cause of why developer time matters more than execution time. The users may need to use newer hardware. But Hardware is cheap – developers are not!
      So lets just waste some CPU cycles instead of precious brain time. At the end of the day, that is the more cost efficient option for the users too.

  8. orbitz

    As Troels pointed out this can be done with the Either type + some monad. I do this in Ocaml with a type called return_value that has a Success and Failure constructor that can take some value. Then it’s easy to write a sequencing function that will only continue on the success portion. It is less pretty than exception code but it has a major benefit: the type system tells me when I forget to handle an exceptional condition. I know when I compile that I am handling every success and error condition (in some way). With a nice sequencing function I get many of the benefits of checked exceptions without the syntactic overload. If a function I am calling adds a new exceptional type, the compiler will tell me (presuming I don’t have any catch-alls in the matching) so I can change my code to handle it. So far I have found this solution fairly adequate, even in Ocaml which does not have any of the monad syntax of Haskell. So, IMO, exceptions have a bad rap for a good reason, they are an accepted version of GOTO. Raymond Chen makes a good argument for why exceptions are a real issue in writing good code too:

    http://blogs.msdn.com/b/oldnewthing/archive/2004/04/22/118161.aspx

    http://blogs.msdn.com/b/oldnewthing/archive/2005/01/14/352949.aspx

    In short, your exception code may look pretty because you think you have are only showing what you want to solve, but it fails for fault tolerant code because you also want to solve error handling. When *anything* can fail it makes it very hard to determine if you have actually caught all the errors.

    1. mortoray

      The discussion of whether exceptions are good or error codes are good appears to actually not relate to the syntax at all. It comes to the argument of whether we should have checked errors or unchecked errors. Error codes and function signature exceptions are checked exceptions, whereas the throw anything approach is unchecked.

      I definitely find value in both. I definitely find fault in both. Somewhere there must be a happy middle-ground or merger of both approaches.

    2. orbitz

      Return codes are not checked exceptions in the standard meaning of the word, the type system does not enforce you handle them. It is just as happy to let you not check the result of an error code as it is to let you not handle an unchecked exception.

      Indeed, like most things in PLT it has little to do with syntax.

  9. Iouri

    int,error_code sum_file( string name )
    {
    ..
    if(err) { goto error; }
    ..
    return (sum, error);
    error:
    return (sum, error);
    }

    1. mortoray

      Yes, if your error handling is not a one-liner the goto approach will make it cleaner. This doesn’t remove the need to do an “if” expression after every function call however — so you haven’t saved a great deal of the bulk. But I do this type of thing in code where error handling is more than one-line, as it prevents the flow from being interrupted too much.

      You should also be concerned that the reason many people are opposed to exceptions is because they attribute them to being nothing more than a “goto” in disguise and hate gotos. Thus to those people, this solution would not be acceptable either.

  10. Andrzej Krzemieński

    Hi, I want to share one observation on teaching exception handling mechanism rather than the mechanism itself.

    Showing that exception handling is useful, or more useful than some other technique, is difficult because it is one of these features whose value is only visible in big scale projects. (You can argue that one can also observe the benefit for small programs, and that would probably be true also, but the full power of the mechanism compared to the cost of the syntax and learning us only observable in big programs, where solutions like return codes become unmanageable.)

    On the other, hand while showing examples you cannot afford to put too big examples of code without risking putting off your readers.

    The benefits of exceptions like the separation of signalling an exception from handling the exception cannot be easily shown. That is that one function (or rather many functions) signals the exception, one function handles the exception, but there are many, many functions in between in the call stack that are exception neutral: they may not even know that something may be thrown “through them”. These in-between functions can be even written in good old C, which doesn’t have exceptions at all. In contrast, with return codes, or combined return values you have to check and dispatch at every level.

    Special monads in Haskell probably also offer the above separation and “exception transparency” (I do not know Haskell), but then, it would also be difficult to show that to the casual reader.

    I have seen an attempt to show that in Java tutorial, but funny enough, in Java no function is exception-transparent.

    1. mortoray

      I think the issue may reside in starting with an imperative style of programming (where many lessons start in many languages). Exceptions are clearly not an imperative construct: they interrupt the primary flow of a program. I guess even if you start with functional programming, the exception would interrupt the normal flow — as suddenly you break out of functions which have not been evaluated.

      One of the big complaints about exception is exactly this flow interruption: it is labelled as a non-local goto. For this reason it is also called unpredictable. This likely relates to your comments about teaching it — if you do not have a solid grasp of what exceptions are doing you may consider their behaviour coutnerintuitive and/or flawed.

      Showing complete examples is always a challenge. It’s a problem any writer struggles with anytime examples are provided. People will inevitably come back and state the example insufficieny to demonstrate the concept, or show better alternatives for that particular example.

    2. orbitz

      Even with a solid grasp of exceptions one can still find them flawed. The lack of type safety that most implementations of exceptions carry can be enough to sour people on them. Haskell/Ocaml/SML provide a means of getting both type safety and compact syntax. In Ocaml I generally only use exceptions for really things that should be fatal, such as things the type system is not powerful enough to protect me from but should never happen in the wild unless something is horribly wrong.

    3. orbitz

      @Andrzej
      For memory I don’t have much option in Ocaml since it is managing memory. In that case it is an exception, which I don’t consider too unreasonable. Running out of memory is something that can happen at any time and things are usually horribly wrong in that case, so that fits with what I described above. Other resources though, such as files, make more sense, to me, to fit into this sum type + monadic functions scheme. Do you disagree?

  11. Andrzej Krzemieński

    @orbitz: I could not agree or disagree with the latter, because I am not familiar with the mechanism using monads. I was just curious. However, I did have doubts about the former, which I could call a “programmer error” – in that case, since this is as fatal as a situation can be, I am used to just terminating the program: throwing an exception appears to me less appropriate because you can just handle the exception and proceed, which would not be correct for a _fatal_ situation.

  12. orbitz

    @Andrzej
    Most GC’d languages seem to raise some kind of ‘out of memory’ exception when it an allocation fails. In C malloc returns NULL for heap allocations, C++ has std::bad_alloc, Java has OutOfMemoryException. It isn’t necessarily *that* fatal, after all. If you try to allocate 30GB and only have 29GB left, then the allocation failing is ok, you still have plenty of room to do other work. With virtual memory the whole situation becomes a bit more complicated, though. I don’t know of any language that gracefully handles local variable allocation failure gracefully (generally on the stack).

    1. Andrzej Krzemieński

      @orbitz: I agree that out-of-mem is best handled by an exception. I just thought that by “things that should be fatal, such as things the type system is not powerful enough to protect me from but should never happen in the wild unless something is horribly wrong” you meant something like programmer error, like storing a negative value for a type that is supposed to represent a distance. I guess I misunderstood you.

    2. orbitz

      The type system can pretty easily protect you against something like that. An example I’m thinking of is more a long the lines of taking the head of a list that should be guaranteed always have an element in it. I don’t believe Ocaml’s type system can protect you against that but the semantics of the program usage may imply that such a thing is impossible. So I guess that is kind of programmer error.

  13. evohunz

    I don’t like exceptions exactly because they make an attempt to hide the possibility of error from the caller.

    In turn, the caller will do nothing about it, because “everything could go wrong and the code calling me should handle it all.”
    This is just ignoring the problem and hopping someone actually remembers to handle it.

    Also, the exception mechanism can hit you exactly where you say it’s useful: useful information. If your function “sum_file” fails at parsing the number, you probably would like to know in what line of the file the problem is. And the parsing function can’t provide this information on the exception it raises. By forcing the caller to handle a failure return type, it’s easier to remember this and add this info.

    1. mortoray

      At application level domains (like web servers, client programs) errors can happen just about anywhere. Libraries have become abstract enough that there is no possibility to know what errors may occur, and it is impossible for each level of code to deal with them.

      Most error handling which is incorrect, that I’ve seen, is because people are attempting to handle an error that they shouldn’t. If you don’t know how to handle an error you should just propagate it. But every level you write has to be error-safe.

    2. metz2000

      @evohunz: The parsing function may not know the line number but can include the text what didn’t parse in exception details. On the other hand, using error codes will not make the parse function aware of line number either.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s