So far things look reasonably good in x86 mode. The code that I have been developing worked fine in Excel 2010 without needing the new SDK at all; which is how it should be given there will be x86 XLLs out there that have been around since Excel v4...

The x64 side of things is looking a little less rosy at this point. It took me a while to set up the x64 builds for my projects and to deal with the inevitable warnings that you get from building as x86 and then switching to x64. That's all been dealt with and the XLL loads nicely into Excel 2010 and works fine right up to the point where it switches to managed code for the first time; during this transition one of Excel's threads goes off on one and pegs a cpu at 100% whilst Process Explorer's stack capture shows that it's somewhere deep inside of CorBindToCurrentRuntime(). I've narrowed the problem down and sent a variation on the 'generic' example XLL back to my Microsoft contact so that they can look at the problem.

So, right now, Excel 2010 x64 support for my managed XLLs is on hold.

The idea behind asynchronous worksheet functions is that some worksheet functions may take a long time to complete and it would be better to run them in the background rather than have them block Excel's recalculations whilst they complete. This means that the Managed XLL needs to manage a thread pool for the asynchronous worksheet function execution and we need to deal with a few issues around passing results back to the sheet. The work that I did on managed RTD servers helped a lot for the asynchronous worksheet functions as I'd already solved the problems of calling into the correct AppDomain from arbitrary threads. Passing results back to the sheet was a bit more thorny; the asynchronous worksheet functions need to be marked as volatile so that Excel will recalculate them whenever a recalculation happens (even if none of their inputs have changed), and the Managed XLL needs to maintain state for each asynchronous call so that it knows if the call is ready to return results, or needs to be run again, or whatever. Finally we need some way of telling Excel to recalculate a given cell when the asynchronous worksheet function is complete; luckily Excel's COM automation interface comes to the rescue here. The result is pretty cool and once I had the basic asynchronous nature of the calls working I added the ability for them to return interim results as well. I still need to do some work to get all of this to work in versions of Excel prior to 2007 but that shouldn't be too hard, I hope.

From an addin developer's perspective asynchronous worksheet functions are as easy as this:

      [AsyncWorksheetFunction]
      static public int AsyncWorksheetFunction1(
         [Optional] [DefaultValue(10000)] int delay)
      {
         Thread.Sleep(delay);
          
         numCalls1++;
    
         return numCalls1;
      }

or maybe this if you want to return interim results or abort a call if requested to do so...

      [AsyncWorksheetFunction]
      static public int AsyncWorksheetFunction2(
         [DefaultValue(10000)] int delay,
         IAsyncWorksheetFunctionCall call)
      {
         for (int i = 0; i < delay; ++i)
         {
            Thread.Sleep(1);
 
            if (call.BreakRequested())
            {
               break;
            }
 
            if (i % 100 == 0)
            {
               call.ReturnInterimResult(i);
            }
         }
 
         numCalls2++;
 
         return numCalls2;
      }

Anyway, during this I came across a slight problem. The architecture is such that I create a custom AppDomain for the managed code that the XLL hosts inside of Excel. The XLL then causes unmanaged callbacks into this managed code when the various Excel RTD COM object functions are called. The bridge between the unmanaged code and the managed code is a gcroot<> that holds onto the managed code inside the unmanaged wrapper via a GCHandle. The COM call eventually calls through this gcroot<> and into the managed code. Unfortunately unmanaged code knows nothing of AppDomains and (it seems from reading this) that an AppDomain is chosen arbitrarily when the unmanaged call into the managed code occurs. This causes the code to fail with a "Cannot pass a GCHandle across AppDomains" exception.

Luckily for me, Miral, over at Thoughts from Mirality has already solved this issue and covers the problem and a solution here. The trick is that you need to use a delegate, which knows about the AppDomain that it relates to, and then call through the delegate by converting it to a function pointer. This effectively marshals the unmanaged call into the correct AppDomain before executing the managed code.

Miral's example code was enough to help me along my way but I stumbled a couple of times before I solved my particular version of the problem so I thought it would be worth documenting what I needed to do to get this to work for me.

Lets say we have the following unmanaged callback. Some code that knows nothing about managed code wants to call the following function:

void Connect(
   const long topicId, 
   const JetByteTools::Win32::StringVector &topics, 
   const bool getNewValue, 
   VARIANT &result);

The first thing we need to do is create a typedef for the function signature and a managed class that can provide us with the delegate that we need to be able to call this correctly. Something like this, perhaps:

typedef void (__stdcall ConnectFnc)(
   const long topicId, 
   const JetByteTools::Win32::StringVector &topics, 
   const bool getNewValue, 
   VARIANT &result);
 
ref class ConnectDelegate
{
   public :
 
      ConnectDelegate();
         :  m_delegate(gcnew Delegate(this, &ConnectDelegate::Connect))
      {
      }
 
      ConnectFnc *GetDelegateFunctionPointer();
      {
         return   (ConnectFnc*)(Marshal::GetFunctionPointerForDelegate(m_delegate).ToPointer());
      }
 
   private :
 
      void Connect(
         long topicId, 
         JetByteTools::Win32::StringVector &topics, 
         bool getNewValue, 
         VARIANT &result)
      {
         // YOUR CODE GOES HERE and is executed in the correct AppDomain...
      }
 
      delegate void Delegate(
         long topicId, 
         JetByteTools::Win32::StringVector &topics, 
         bool getNewValue, 
         VARIANT &result);
 
      Delegate ^m_delegate;
};

The managed code that you want to execute inside the correct AppDomain lives in ConnectDelegate::Connect(). The constructor of our class simply creates the delegate to call the function and GetDelegateFunctionPointer() does exactly what it says on the tin. An instance of the ConnectDelegate class needs to be created by code that is running inside the AppDomain that you want your unmanaged callback to call into.

It might look like you can then simply call ConnectDelegate::GetDelegateFunctionPointer() from your unmanaged code to obtain a function pointer and make your callback, but unfortunately you need one further level of indirection before that's possible. As I said at the start, the problem is that calling through a GCHandle via a gcroot<> from unmanaged code gives a "Cannot pass a GCHandle across AppDomains" exception. Well, right now you still need a gcroot<> to hold onto your ConnectDelegate so that you can call it from unmanaged code and calling ConnectDelegate::GetDelegateFunctionPointer() calls through that GCHandle and raises the exact same exception... The trick is that you also need to call ConnectDelegate::GetDelegateFunctionPointer() from within your target AppDomain and then store away the function pointer for later use. Since this leaves you with a gcroot<ConnectDelegate^> and a ConnectFnc* to hold onto in unmanaged code I put together a simple unmanaged class which manages both of these and provides an overloaded function call operator to make it easy to call the ConnectFnc*. The result is something like this:

class ConnectFunction
{
   public :
 
      ConnectFunction()
         :   m_pDelegate(0),
             m_pFunction(0)
      {
      }
 
      ~ConnectFunction()
      {
         delete m_pDelegate;
      }
 
      void operator()(
         const long topicId, 
         const JetByteTools::Win32::StringVector &topics, 
         const bool getNewValue, 
         VARIANT &result) const
      {
         m_pFunction(topicId, topics, getNewValue, result);
      }
 
      void Setup()
      {
         m_pDelegate = new gcroot<connectdelegate^>(gcnew ConnectDelegate());   
 
         m_pFunction = (*m_pDelegate)->GetDelegateFunctionPointer();
      }
 
   private :
 
      gcroot<ConnectDelegate ^> *m_pDelegate;
 
      ConnectFnc *m_pFunction;
 
      // No copies do not implement
      ConnectFunction(const ConnectFunction &rhs);
      ConnectFunction &operator=(const ConnectFunction &rhs);
};

You can then create an instance of this class and then call Setup() from within your target AppDomain. You can then use the function call operator on your instance from within your unmanaged code to make the callback. You can move the body of Setup() into the constructor if you like, my particular situation requires two stage construction.

I've just completed phase 2 of this project and it now works just like we originally envisaged. Managed code can be decorated with custom attributes and the XLL uses reflection to work out which classes within an assembly are being exposed to Excel. It then looks for [WorksheetFunction] attributes and exposes individual functions to Excel whilst working out all of the marshalling requirements itself. Additional attributes allow you to specify that some parameters are optional or have default values. More attributes allow for the definition of custom Excel menus and the provision of the code behind them. By annotating a suitable constructor you can have an IManageAddins interface passed into your managed class when it's constructed and this lets you call back into the XLL framework and/or Excel to do things like enable and disable menu items, etc. Several custom types allow us to pass ranges of Excel cells into and out of managed code, and our standard marshalling can deal with converting Excel ranges that contain consistent types to arrays of native managed types. Volatile and command equivalent worksheet functions are supported and, all in all, it all seems to work pretty nicely.

When the XLL is loaded it reads its configuration from a config file and loads the specified assemblies, parses them for attributes, builds and registers the appropriate data structures and wires up the XLL entry points to the appropriate marshaller and from there into managed code.

The development work so far has been quite interesting. I'm building a prototype XLL style Excel addin that allows worksheet functions to be developed in C#. There are commercial products out there that do this, but my client wants me to write one for them. This takes me back to some work that I did for a previous banking client where I wrote some XLLs that allowed them to talk to their back-end servers from Excel using CORBA; in that system the XLLs were all in C++, this time around the idea is that the 'meat' of the XLL functionality will be in managed code.

The past few days have been interesting. My initial prototype was a C++ dll that hosted the CLR. This was, in retrospect, always going to have problems. The CLR can only be hosted once within a process and, once loaded, it can't be unloaded. XLLs by their very nature can be unloaded and reloaded and "other things" could cause Excel to host the CLR itself... The initial prototype did, at least, allow me to get a thin thread of functionality all the way through from Excel to managed code and back, but it quickly became apparent that it was a dead-end.

The current prototype involves a straight C++ XLL which dynamically loads a C++/CLI dll which then configures a new AppDomain and loads the managed addin code. Since one of my client's requirements is "xcopy deployment" (and the ability to configure the managed XLL loader to load numerous managed addins) the first problem to overcome was that of loading code into a new AppDomain when the new AppDomain had a different assembly search path to the default domain. Due to the way communication between AppDomains is managed this was a little more complex than I expected.

To load an assembly into a new AppDomain and then execute code in that AppDomain you need to also load the assembly that contains the types that you're accessing in the new AppDomain into the AppDomain that's doing the loading... Well, you do if you want to use a class that's derived from MarshalByRefObject and which uses transparent proxies to talk across AppDomain boundaries.

The recommended way to do things, with code like this:

      AppDomainSetup ^setup = gcnew AppDomainSetup();
      
      setup->ApplicationName = "My App";
      setup->ApplicationBase = gcnew String(GetModulePathName(hInst));
  
      AppDomain^ newDomain = AppDomain::CreateDomain( "newDomain", nullptr, setup );
  
      Object ^obj = newDomain->CreateInstanceFromAndUnwrap(
         "MyAssembly",
         "MyType");
  
      MyType ^myType = (MyType^)obj;
  
      myType->DoThing();

requires that "MyAssembly" is loaded into both the newly created AppDomain (as that's where you want to run the code!) and also into the current AppDomain (as "MyType" is needed by the transparant proxy to allow you to talk across AppDomain boundaries to the code in the new AppDomain.

Since the default AppDomain has an assembly search path that's different to the assembly search path for the new AppDomain we have to somehow make the required assembly available to both AppDomains.

My first step was to include the required type in the assembly that's doing the AppDomain creation. I figured that since this assembly was already loaded in the AppDomain that things would "just work". Unfortunately they didn't. The code above resulted in an exception when the cast from obj to MyType took place; "Unable to cast transparent proxy". Changing the cast to a dynamic_cast solved that problem but didn't actually work, the resulting object was in the wrong AppDomain.

It seems that even though the assembly that contained the type that was required was already loaded in the AppDomain it was still loaded again during the cast (possibly because of some identity assembly issue due to the fact that the executing assembly was loaded as a C++ dll which just happens to contain managed code...). Anyway. The assembly loader was looking to load the assembly again and when it did it treated the two copies of the assembly that it had loaded as different and the types were not considered to be equal... Which makes sense, in a frustrating kind of way.

After banging my head on the desk for a while I came up with a solution. I needed to get involved with the assembly loading for the current AppDomain and registering a ResolveEventHandler for the AppDomain's AssemblyResolve event could allow that... I plugged in the event handler and tried to run the code again, the handler was called and the full name of my assembly was passed in... By implementing my event handler like this:

ref class Resolver
{
   public :
  
      static Assembly ^Resolve(
         Object ^sender, 
         ResolveEventArgs ^args)
      {
         if (args->Name == Assembly::GetExecutingAssembly()->FullName)
         {
            return Assembly::GetExecutingAssembly();
         }
  
         return nullptr;
      }
};

I could return the current assembly when asked for it, and prevent another copy being loaded... This seemed to be enough to get the code to work...

      AppDomain::CurrentDomain->AssemblyResolve += gcnew ResolveEventHandler(Resolver::Resolve);
  
      AppDomainSetup ^setup = gcnew AppDomainSetup();
      
      setup->ApplicationName = "My App";
      setup->ApplicationBase = gcnew String(GetModulePathName(hInst));
  
      AppDomain^ newDomain = AppDomain::CreateDomain( "newDomain", nullptr, setup );
  
      Object ^obj = newDomain->CreateInstanceFromAndUnwrap(
         Assembly::GetExecutingAssembly()->Location,
         "MyType");
  
      MyType ^myType = (MyType^)obj;
  
      myType->DoThing();

Will result in DoThing() being executed in the new AppDomain.