Defective Language

Cannot create a proper clone method with shared_ptr

Smart pointers in C++ are nice, yet fraught with irregularities. One is the inability to create a proper clone function. This requires a feature called covariant return types, which C++ supports, but only for pointers. That sounds okay, but despite their name, smart pointers are not actually pointers.

Covariant return

A prototypical example of covariant return types is a clone method. Reasonably, we expect that a ‘clone’ method returns the type on which it was called. It wouldn’t make sense to call it ‘clone’ if it could return an incompatible object type.

B * b = create_b();

//with covariant returns
B * dup = b->clone();

//without covariant returns
B * dup = static_cast<B*>(b->clone());

The above covariant call can be implemented as below. Here we see a derived class overriding the virtual function with a different return type. This is well defined so long as the return is a derived type.

class A {
	virtual A * clone();
}
class B : public A {
	virtual B * clone();
}
class C : public B {
	virtual C * clone();
}

The caller of clone is returned the same type without having to cast it. The virtual function still works as expected: if you have a ‘B*’ to an object of type ‘C’, the ‘C::clone’ method will be called and its return cast to a ‘B*’. Beyond just a convenience, it is safer to have the compiler do this than rely on a ‘static_cast’. As code changes over time, one of those ‘static_cast’ statements will inevitably perform an invalid cast.

Broken with shared_ptr

I consider ‘shared_ptr’ to be a good option for general purpose object management. Unfortunately it isn’t a real pointer. Only via a series of implicit conversion functions does it behave somewhat like a pointer. Weaknesses show up in many situations: covariant returns are one of them. It is not possible to code a ‘clone’ function as expected.

struct A {
	virtual shared_ptr<A> clone() {
		return make_shared<A>(*this);
	}
};

struct B : public A {
	shared_ptr<B> clone() {
		return make_shared<B>(*this);
	}
};

Covariant returns are restricted to true pointer types. The above code results in a compiler error. Using a ‘smart_ptr’ requires returning the same type in all overrides of the virtual function. Using these non-covariant clone methods requires an unsafe cast.

shared_ptr<B> b = create_b();

//requires this unsafe syntax
shared_ptr<B> ugh = static_pointer_cast<B>( b->clone() );

//the preferred syntax
shared_ptr<B> good = b->clone();

Using macros and/or other template techniques the overhead can be reduced somewhat. But it’s code I shouldn’t have to write. Indeed, a minor error in one of my versions resulted in a hard-to-find defect. It’s a defect that happened because a new C++ feature, smart pointers, are effectively incompatible with a previous feature, covariant returns. I’m in favour of new language features, but they should expand on previous features, not deprecate them. As it stands, a proper clone function using smart pointers cannot be written.

17 replies »

    • As a user, yes, that’s exactly what I want. As a compiler vendor this is tricky. I think covariant is trivial now since all those pointers have the same value: at runtime the notion of covariance is lost. Or is there already a sitaution where some kind of thunking actually has to happen? If yes, then I see no reason why not to allow user conversions.

    • “Or is there already a sitaution where some kind of thunking actually has to happen?”
      — If this question was directed to me: I do not know the answer, and I do not have the compiler writer’s perspective. Not sure if this is on topic, but one of the participants in the referred discussion mentioned the following:

      “When using gcc, the vtable entry for f actually points to a compiler generated thunk that performs the return type conversion from C* to B*. And I guess the proposal is to allow compiler generated thunks to perform user-defined conversions”

    • That comment was what got me thinking. I’m wondering if the current covariants are easy to implement simply because they all actually have the same address — it is merely a type feature of the language.

      The question is whether multiple inheritence can produce a situation now where the base/derived classes, as returned by a covariant virtual function, can have different addresses.

    • If I understand your question correctly, the answer is “yes”:

      struct B1 { int i1; };
      
      struct B2 { int i2; };
      
      struct D : B1, B2 { };
      
      int main()
      {
      	D d;
      	B1 * b1 = &d;
      	B2 * b2 = &d;
      
      	assert (reinterpret_cast<B1*>(&d) == b1);
      	assert (reinterpret_cast<B2*>(&d) != b2); // different address
      }
      
    • Ah yes, and there could be a covariant return function in both B1 and B2. The relevance is that the compiler does indeed have to generate thunking code around any such functions. That is, depending on through which type you call, and the actual type, there need to be several functions involved and no direct dispatching to the target function. I can’t see that it’d matter much what that thunking code would do; it shouldn’t be any harder to allow user conversions.

      Do you know of an open request C++ that would allow them?

  1. struct A {
        virtual shared_ptr<A> clone() ;
            return make_shared<A>(*this);
        }
    };
    

    Shouldn’t this be:

    struct A {
        virtual shared_ptr<A> clone() 
        {
            return make_shared<A>(*this);
        }
    };
    
    • Most languages and libraries involve some degree of thunking. It’s a rather generic term applied to the “interception” of functions calls. That is, whenever somethign happens in between call site and the destination function we tend to call it thunking.

  2. C++11 emphasizes free functions, which solve this problem:

        struct A {
            virtual shared_ptr<A> clone_impl() {
                return make_shared<A>(*this);
            }
        };
         
        struct B : public A {    
            virtual shared_ptr<A> clone_impl() {
                return make_shared<B>(*this); // shared_ptr can implicitly cast to shared_ptr
            }
        };
        
        template<typename T>
        shared_ptr clone(shared_ptr<T> p)
        {
            return static_ptr_cast<T>(p->clone_impl());
        }
        
        shared_ptr<B> b = create_b();
          
        shared_ptr<B> cloned_b = clone(b);
    

    You could improve on this with static_assert or a debug check using dynamic shared_ptr casting rather than static, etc, but the general idea is the same.

    • This is certainily a solution, but it is only a partial solution. The problem is that the clone_impl functions aren’t forced to return the correct type. They can still return any type which is dervied from A. The static_cast fails silently and you get random memory corruption. Adding a dynamic cast/check may be a performance issue at this point, and shouldn’t be required.

      But yes, this approach at least centralizes the static_cast and isolates the potential problem. I tend to use this, and variants, when I encounter the problem.

  3. Here is how to do it. See http://ideone.com/Bc3vw6 for a working example.

    You need 2 things.

    1) a virtual clone_impl thatreturns shared_pointer. This should be protected.

    2) A non-virtual clone function that returns shared_pointer

    Every class in the hierarchy should have these 2 functions defined. The virtual clone_impl makes sure that the clone is done without slicing. The non-virtual clone function handles the “covariant” part by hiding the Base class clone with its own clone.

    You can make this easier with a template that you derive from using CRTP that provides these functions automatically. Take a look at the clonable template in the above code.

    Mortoray what do you think of this solution?

    • It looks promising, albeit complicated. At the moment it doesn’t appear to support a deeper hierarchy, you’d need at least one more template parameter. You need both the immediate base class and the result of clone_impl. Consider where C derives from B derives from A. I think it’d work, and be a bit safer, but declaring the hierarchy now is quite verbose.

  4. Using CRTP may solve this in some cases:

    template
    struct Base {
    shared_ptr clone() const
    {
    return make_shared(*this);
    }
    };

    struct Derived : public AbstractBase {
    // hooray – clone is already made for us
    };

    Of course this prevents storing a generic pointer to Base & cloning that, but it may solve some problems.

    • WordPress swallowed your template code, but I guess you have something like this:

      template<typename T>
      struct AbstractBase {
         shared_ptr<T> clone() { ... }
      };
      
      struct Derived : public AbstractBase<Derived> {
      };
      

      But I’m not sure how this solves the problem of covariance. The real base class still needs a `clone` function, but that will conflict with the clone of the derived classes.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s