This allows me to load a set of applications that can be run when the server starts up and start them. I can then tell each network endpoint the name of the application that connections are to be routed to and as each new connection comes in the socket server that is managing that end point can look up the application by name in the CApplicationCollection and connect the connection to it. If the application is not running then the connection will fail and be closed after issuing an error message.

Since there's a difference between applications that the server CAN run and applications that ARE running I can easily restart an application by locating the running instance, telling it we're about to shut down and then unloading its application domain. Meanwhile I can create a new application domain, start a new copy of the managed application, based on the application details that are stored for the application, and insert it into the running applications collection. In fact I do this in reverse, starting and inserting the new application and then shutting down the old. This minimises the time when an application isn't available.

Whenever an application is stopped I need to be able to walk the list of connections that are currently connected to the application and terminate them. This requires that the unmanaged application objects maintain a collection of active connections. This deals with the 'kill' style restarts that I spoke of in the previous article. To handle the 'kind' restarts I need to be able to keep a running application around after it has been removed from the running applications collection and only unload its application domain when all currently active connections have been completed. To enable this the application object is reference counted and the application domain is unloaded when the application object is deleted. This allows me to have each active connection hold a reference to the application that it is connected to and also have the CApplicationCollection hold a reference whilst it has the application in its collection of running applications. When the application is restarted it is removed from the running applications collection and the reference owned by the CApplicationCollection is released, now only the active connections are keeping the application alive and when the last of those ends the application domain is unloaded. Meanwhile we've already started a new copy of the application and placed it in the running applications collection and so new connections go to the new copy of the application and existing connections continue in the old copy.

Automatic restarts are supported by enabling shadow copying on the application domains so that the assemblies aren't locked whilst the application is running and adding a directory change monitor to look for updated files. The directory change monitor starts a 10 second timer when it spots a file change and when the timer expires the application is restarted. The server demonstrates both kind and kill automatic restarts. The timer is necessary to prevent multiple restarts when files are updated by hand, ideally you set a timer for long enough to allow someone to copy or edit files and when the "quiet period" has expired the application is restarted.

Finally a management socket server presents a simple text based protocol that's accessible via telnet which allows the user to list and manipulate the applications. Right now you can't add new applications and endpoints to a running server but there's nothing in the architecture to prevent it. Likewise you can't remove endpoints but it could be added.

The only thing that you really need to do in your custom AppDomainManager registered is make sure that you implement InitializeNewDomain() so that you set the flags to tell the CLR to register new application domains with your unmanaged host.

      public override void InitializeNewDomain(AppDomainSetup appDomainInfo)
      {
         // let the unmanaged host know about us
         InitializationFlags = AppDomainManagerInitializationOptions.RegisterWithHost;
      }

With that set you'll get a call into your IHostControl::SetAppDomainManager() method for each application domain that's created. To be able to do anything interesting with your hosting powers you probably want to create an interface for communicating between your unmanaged host and your managed AppDomainManager, and possibly one for communicating back the other way. I find it easiest to create the interfaces in the old school COM way in an IDL file that is compiled into a type library and then imported into an assembly. For me this gives me all that I want without any of the cruftiness of the other ways of doing this. The IDL from my example server for its AppDomainManager looks like this.

   [
      object,
      uuid(0EC4D543-008A-4cd3-88DB-A5199F416251),
      helpstring("IManagedHost Interface"),
      pointer_default(unique)
   ]
   interface IManagedHost : IUnknown
   {
      HRESULT InitialiseDefaultAppDomain(
         [in] IUnmanagedHost *host);
  
      HRESULT CreateApplicationInNewAppDomain(
         [in] BSTR appDomainName,
         [in] BSTR applicationName,
         [in] BSTR applicationDirectory,
         [in] BSTR sharedDirectory,
         [in] BSTR configFileName,
         [in] BOOL enableShadowCopy,
         [out, retval] long *pDomainID);
  
      HRESULT Start(
         [in] BSTR assemblyName,
         [in] BSTR typeName,
         [in] BSTR applicationName,
         [out, retval] IManagedApplication **application);
  
      HRESULT RequestStop();
  
      HRESULT Stop();
   };

I then implement this in my managed AppDomainManager and QI for it when IHostControl::SetAppDomainManager() is called.

HRESULT CManagedHost::SetAppDomainManager(
  DWORD dwAppDomainID,
  IUnknown *pUnkAppDomainManager)
{
   ICriticalSection::Owner lock(m_criticalSection);
  
   DomainManagerMap::iterator it = m_domainManagers.find(dwAppDomainID);
  
   if (m_domainManagers.end() != it)
   {
      it->second = SafeRelease(it->second);
   }
  
   IManagedHost *pHost = 0;
  
   const HRESULT hr = pUnkAppDomainManager->QueryInterface(IID_IManagedHost, (void**)&pHost);
  
   if (SUCCEEDED(hr))
   {
      m_domainManagers[dwAppDomainID] = pHost;
   }
  
   return hr;
}

When writing a system to load and unload plugins (or applications) you need to be aware that you can't unload the "default application domain". In effect there's the "default application domain" and all the other application domains (secondary application domains). Plugin code should be loaded into a new application domain so that it's isolated and so that it can be unloaded without needing to restart the hosting process. Since you can only have one type as an AddDomainManager for all application domains this can cause the custom AppDomainManager to have two distinct roles. The first is to be the manager of the default application domain. In this role it's responsible for creating new application domains. The second role is to be the manager of the secondary application domains, the ones which run the plugin code. This second role allows you to load your plugin code into your new application domains and control the execution of your plugins from unmanaged code.

In my example IDL above Start(), RequestStop() and Stop() are methods that can be called only in secondary application domains and InitialiseDefaultAppDomain() and CreateApplicationInNewAppDomain() must only be made onto the default application domain.

InitialiseDefaultAppDomain() is called explicitly just after I start the CLR and set its error escalation policy. It allows me to pass an interface to the managed part of my hosting code that allows it to communicate with the unmanaged part, this interface is pretty simple.

   [
      object,
      uuid(AFF50626-DDCF-481c-9CE7-0F95B0E9E6D7),
      helpstring("IUnmanagedHost Interface"),
      pointer_default(unique)
   ]
   interface IUnmanagedHost : IHostLog
   {
      HRESULT ShutdownHost();
   }
  
   [
      object,
      uuid(7F153933-8BE3-45b6-B8C5-22B5EFA0B8FD),
      helpstring("IHostLog Interface"),
      pointer_default(unique)
   ]
   interface IHostLog : IUnknown
   {
      HRESULT LogMessage(
         [in] BSTR message);
   }

Strictly for demonstration purposes it allows the managed code to write to the same log file as my unmanaged code and it also allows the managed code to request the unmanaged host shut down. The initialise function also lets me set up event handlers for the default application domain (and for demonstration purposes these event handlers can use the log interface to keep me informed of what's going on).

Loading and running plugin code in the CLR is a two stage process. First I call CreateApplicationInNewAppDomain() on the default AppDomainManager which creates a new application domain and sets it up ready to run the plugin code.

      int IManagedHost.CreateApplicationInNewAppDomain(
         string appDomainName, 
         string applicationName, 
         string applicationDirectory, 
         string sharedDirectory,
         string configFile,
         int enableShadowCopy)
      {
         if (!defaultAppDomain)
         {
            throw new InvalidOperationException("Should only be called on the default application domain, not on \"" + this.appDomainName + "\"");
         }
  
         if (host == null)
         {
            throw new InvalidOperationException("Not initialised, call InitialiseDefaultAppDomain() first!");
         }
  
         AppDomainSetup setup = new AppDomainSetup();
  
         setup.ApplicationName = applicationName;
         setup.ApplicationBase = ".";
         setup.PrivateBinPath = ".;" + applicationDirectory + ";" + sharedDirectory;
         setup.ConfigurationFile = configFile;
  
         if (enableShadowCopy == 1)
         {
            setup.ShadowCopyFiles = "true";
            setup.CachePath = ".\\Cache\\" + applicationDirectory;
         }
  
         AppDomain appDomain = AppDomain.CreateDomain(appDomainName, null, setup);
  
         ManagedHost manager = (ManagedHost)appDomain.DomainManager;
  
         manager.SetupAppDomain(appDomainName, appDomain, host);
  
         return appDomain.Id;
      }

Note that once the default AppDomainManager has created a new application domain by calling CreateDomain() it can access the AppDomainManager of the new application domain and, in my case, initialise it. This is similar to what happens with the unmanaged code explicitly initialises the default AppDomainManager by calling InitialiseDefaultAppDomain().

Once the new application domain has been created I can call Start() on its AppDomainManager, this simply loads the supplied assembly into the new application domain, creates an instance of the type that is the entry point for my application code, calls OnApplicationStart() on it to start the application, and returns interface to the application to unmanaged code so that it can be manipulated from there.

      IManagedApplication IManagedHost.Start(
         string assemblyName,
         string typeName,
         string applicationName)
      {
         if (defaultAppDomain)
         {
            throw new InvalidOperationException("Should only be called on the application domain that was created for application \"" + application + "\", not on the default application domain");
         }
  
         if (appDomainName != applicationName)
         {
            throw new InvalidOperationException("Should only be called on the application domain that was created for application \"" + application + "\", not on \"" + appDomainName + "\"");
         }
  
         Assembly assembly = Assembly.Load(assemblyName);
  
         object obj = assembly.CreateInstance(typeName);
  
         if (obj == null)
         {
            throw new TypeLoadException("Object \"" + typeName + "\" not found in assembly \"" + assemblyName + "\"");
         }
  
         this.application = (IManagedApplicationControl)obj;
  
         return application.OnApplicationStart(applicationName, host);
      }

My application interface looks like this:

   [
      object,
      uuid(DE2B37FB-190A-4ef9-9CD9-97CE94C3D5FE),
      helpstring("IManagedApplicationControl Interface"),
      pointer_default(unique)
   ]
   interface IManagedApplicationControl : IUnknown
   {
      HRESULT OnApplicationStart(
         [in] BSTR applicationName,
         [in] IHostLog *log,
         [out, retval] IManagedApplication **application);
  
      HRESULT OnApplicationStopRequested();
  
      HRESULT OnApplicationStop();
   };
  
   [
      object,
      uuid(19A68076-9C7E-4981-BE97-949803AEB76F),
      helpstring("IManagedApplication Interface"),
      pointer_default(unique)
   ]
   interface IManagedApplication : IUnknown
   {
      HRESULT OnConnectionEstablished(
         [in] ISocket *socket);
  
      HRESULT OnReadCompleted(
         [in] SAFEARRAY( BYTE ) data,
         [in] ISocket *socket);
  
      HRESULT OnConnectionClientClose(
         [in] ISocket *socket);
   };

The IManagedApplicationControl interface is used from within the custom AppDomainManager and allows it to start and stop the application. The IManagedApplication interface represents the interface that is presented to the unmanaged code. In this simple server example we pass some network events through from the unmanaged server host.

The simple example implementation looks something like this.

   class ManagedApplication : IManagedApplicationControl, IManagedApplication
   {
      private string name;
 
      private IHostLog log;
 
      private void LogMessage(string message)
      {
         log.LogMessage(name + ": " + message);
      }
 
      IManagedApplication IManagedApplicationControl.OnApplicationStart(
         string applicationName, 
         IHostLog log)
      {
         this.name = applicationName;
 
         this.log = log;
 
         LogMessage("OnApplicationStart");

         return this;
      }
 
      void IManagedApplicationControl.OnApplicationStopRequested()
      {
         LogMessage("OnApplicationStopRequested");
      }
 
      void IManagedApplicationControl.OnApplicationStop()
      {
         LogMessage("OnApplicationStop");
      }
 
      void IManagedApplication.OnConnectionEstablished(
         ISocket socket)
      {
         socket.WriteString("Welcome to CLR echo server: " + name + "\r\n");
 
         socket.Read();
      }
 
      void IManagedApplication.OnReadCompleted(
         byte[] data,
         ISocket socket)
      {
          socket.Write(data);
 
          socket.Read();
      }
 
      void IManagedApplication.OnConnectionClientClose(
         ISocket socket)
      {
         LogMessage("OnConnectionClientClose");
      }
   }

As network events occur the are passed through to the appropriate managed application which can deal with them. The unmanaged side of the call is simply a COM method call on an IManagedApplication instance.

I have two ways to stop an application. Firstly I can call IManagedApplicationControl::Stop() from the applications AppDomainManager. I call this just before I unload the application domain, the idea being that it allows the plugin code to clean up before it is destroyed. Secondly I can call IManagedApplicationControl::RequestStop(), this exists so that my hosting code can demonstrate immediate application domain clear down and 'once all existing connections have completed' application domain clear down. With plugin code that is accessed via network users you may want to restart an application (to upgrade it, for example) whilst there are existing users connected and allow those existing users to continue using the old application until they disconnect, OR you might want to just forcibly abort any existing connections and restart the application domain immediately. I call IManagedApplicationControl::RequestStop() if I'm going to allow the application domain to run until existing users have disconnected, this allows the managed application to react accordingly, such as suggesting to the users that they log off, or whatever.

Once I've decided to stop an application and all existing connections have terminated the AppDomain can be unloaded. I do this by calling ICLRRuntimeHost::UnloadAppDomain() and passing it the ID that was returned when we created the new application domain.

Since the CLR can't be stopped by a host without terminating the host process (and that's by design...) you need to be aware that any of the code that you have plugged into the CLR by way of the hosting interfaces can be called into during process shutdown. Since most of the code that you plug in will be in the form of COM objects you can rely on the COM reference counting to deal with cleaning up the objects (if the CLR could be relied on to release the references at any point...) but since you're now writing code that out-lives main() you have to be intimately aware of how this code uses other code that might rely on module level static variables, and other nastiness. Sure you wouldn't want to be relying on module level statics and the order of creation and destruction of those statics (which, as we know, is undefined across multiple modules) but if you're using the version of STL that ships with VS2005 in any of your hosting COM objects then you are...

My implementation if IHostTaskManager uses a std::map to keep track of its tasks. During process shutdown the CLR calls into my task manager to put a task to sleep, to do this I need to locate the task in my map and doing that relies on iterating the STL map and that, if _HAS_ITERATOR_DEBUGGING is enabled, seems to manipulate a global lock... Unfortunately the global lock has been destroyed by the time the CLR decides to shutdown ... Fortunately the global lock is kept alive by a reference counting class called _Init_locks which, although apparently undocumented, can be used to keep the locks that you need to use alive until the object that needs them is destroyed... So, by sticking an instance of _Init_locks into my task manager class I can make sure that it can iterate its collection after main() has returned and the process is shutting down...