<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>Rambling Comments</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/" />
    <link rel="self" type="application/atom+xml" href="http://www.lenholgate.com/blog/atom.xml" />
    <id>tag:www.lenholgate.com,2010-12-10:/blog//12</id>
    <updated>2010-12-22T13:03:44Z</updated>
    
    <generator uri="http://www.sixapart.com/movabletype/">Movable Type Pro 5.12</generator>

<entry>
    <title>Sinking connection points in C++ objects.</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2002/05/sinking-connection-points-in-c-objects.html" />
    <id>tag:www.socketframework.com,2002:/blog//12.142</id>

    <published>2002-05-30T00:00:00Z</published>
    <updated>2010-12-22T13:03:44Z</updated>

    <summary>Sometimes it would be nice to be able to use a simple C++ object as a sink for Connection Point notifications. This article shows you how....</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        Sometimes it would be nice to be able to use a simple C++ object as a sink for Connection Point notifications. This article shows you how.
        <![CDATA[<p><b>Sinking connection points in C++ objects.</b><br />
Many COM objects provide events to their clients by means on Connection Points. A Connection Point is a generalised method of allowing clients to register a COM interface so that the object can notify them of things that they may be interested in. The problem is, sometimes you will be developing code that simply uses COM objects and doesn't implement any COM objects itself, yet you still need to be able to register a COM object to recieve notifications via a Connection Point. This article shows you how you can use Connection Points and sink events from them without needing to use ATL or MFC or expose real COM objects from your application.</p>

<p><b>Registering for notifications</b><br />
Suppose we want to use a COM object that provides notifications via Connection Points, the NetMeeting Conference Manager object for example. Our application will create an instance of the Conference Manager object and manipulate it via its interface, we will monitor the events it generates by registering a connection point to receieve events generated by the <code>INmManagerNotify</code> interface. The <code>INmManagerNotify</code> interface is shown below:</p>


<pre class="brush: cpp gutter: false">   interface INmManagerNotify : IUnknown
   {
      typedef [unique] INmManagerNotify *LPNMMANAGERNOTIFY;
   
      HRESULT NmUI(
         [in] CONFN uNotify);
  
      HRESULT ConferenceCreated(
         [in] INmConference *pConference);
  
      HRESULT CallCreated(
         [in] INmCall *pCall);
   }</pre>


<p>We don't want to use MFC or ATL and the resulting program will be a console application using straight C++ and COM client APIs. Our problem is that our application simply uses COM, it doesn't expose any COM functionality itself and we don't want to add the complexity of being able to serve real COM objects simply so that we can consume events from a Connection Point. What we'd like to do is simply implement the methods of the notification interface and ask the Conference Manager object to call us when notifications occur without worrying about the necessary plumbing.</p>

<p><b>A reusable, self contained, notification class</b><br />
The work involved to hook up a connection point is fairly straight forward and MFC and ATL could both provide some help if we were using them, but since we're not we have to do the work ourselves. Given the <code>IUnknown</code> of a connectable object we need to obtain a pointer to its <code>IConnectionPointContainer</code> interface and from that obtain a pointer to the required connection point interface. Once we have that, we simply call the <code>Advise()</code> member function, pass our sink interface and keep hold of the cookie that is returned as we need it to disconnect. If we wrap this work in a class we can make sure that we disconnect in the destructor. The class might look something like this:</p>


<pre class="brush: cpp gutter: false">   class CNotify 
   {
      public :
  
         CNotify();
         ~CNotify();
  	   
         void Connect(
            IUnknown *pConnectTo, 
            REFIID riid, 
            IUnknown *pIUnknown);
  
         void Disconnect();
  
         bool Connected() const;
  
         private :
  
            DWORD m_dwCookie;
            IUnknown *m_pIUnknown;
            IConnectionPoint *m_pIConnectionPoint;
            IConnectionPointContainer *m_pIConnectionPointContainer;
   };</pre>


<p>Connecting can be as simple as calling <code>Connect()</code> and passing the connectable object, the IID of the connection point that we wish to connect to and our sink interface. Disconnecting is equally easy.</p>

<p>The class above is useful but it doesn't solve our problem. We still need to have a COM object to pass as our sink interface and we don't have one, or want to build one. However, it's relative easy to put together a template class that's derived from our <code>CNotify</code> class and fills in the missing functionality. The template looks something like this:</p>


<pre class="brush: cpp gutter: false">   template &lt;class NotifyInterface, const GUID *InterfaceIID&gt; class TNotify : 
      private CNotify, 
      public NotifyInterface
   {
      public :
  
         void Connect(IUnknown *pConnectTo)
         {
            CNotify::Connect(pConnectTo, *InterfaceIID, this);
         }
  
         using CNotify::Disconnect;
         using CNotify::Connected;
  
         // IUnknown methods
         ULONG STDMETHODCALLTYPE AddRef()
         {
            return 2;
         }
  
         ULONG STDMETHODCALLTYPE Release()
         {
            return 1;
         }
  
         HRESULT STDMETHODCALLTYPE QueryInterface(REFIID riid, PVOID *ppvObj)
         {
            HRESULT hr = S_OK;
  
            if (riid == IID_IUnknown)
            {
               *ppvObj = (IUnknown *)this;
            }
            else if (riid == *InterfaceIID)
            {
               *ppvObj = (NotifyInterface *)this;
            }
            else
            {
               hr = E_NOINTERFACE;
               *ppvObj = NULL;
            }
  
            return hr;
        }
   };</pre>


<p><code>TNotify</code> is used as a base class for your notification sink object, it derives from the sink interface that you want to implement and provides the <code>IUnknown</code> methods for you. You simply have to implement the sink interface methods and then call <code>Connect()</code> and pass the connectable object that you wish to connect to. Your sink object<br />
might look like this:</p>


<pre class="brush: cpp gutter: false">   class MyConferenceManager : private TNotify&lt;INmManagerNotify, &amp;IID_INmManagerNotify&gt;
   {
      public :
   
         HRESULT STDMETHODCALLTYPE NmUI(CONFN confn)
         {
            return S_OK;
         }
   
         HRESULT STDMETHODCALLTYPE ConferenceCreated(INmConference *pConference)
         {
            return S_OK;
         }
   
         HRESULT STDMETHODCALLTYPE CallCreated(INmCall *pCall)
         {
            return S_OK;
         }
   };</pre>


<p>You need to be aware of a few things when using your sink object. The first is that the COM object that is constructed for you by the <code>TNotify</code> base class expects to reside on the stack, or, at least, it doesn't expect COM to manage its lifetime (notice the implementation of <code>AddRef()</code> and <code>Release()</code>). Secondly the sink object maintains references to the connectable object's interfaces, so that it can disconnect when you call <code>Disconnect()</code> or, if you don't, when it gets destroyed. Finally, since the sink object could make COM calls when it is destroyed you need to make sure that COM is still initialised at that point - so no calling <code>CoUninitialize()</code> before you destroy your sink object.</p>

<p><b>The example code</b><br />
The example code is slightly more complex than that shown above. Because of all the type mapping that you tend to have to do to call COM methods from straight C++ we tend to wrap the COM objects in thin C++ wrappers. We do this with the interface that we obtain for the <code>INmManager</code> object and we use the same wrapper to sink the events. This leaves us with a single C++ object that exposes both the <code>INmManager</code> interface which translates the argument types to C++ friendly types and throws exceptions on method failures, and also exposes the <code>INmManagerNotify</code> interface to notify us of events that occur on the object. As you'll see, creating and using the NetMeeting Conference Manager COM object and wiring it up to receieve notification events is as simple this:</p>


<pre class="brush: cpp gutter: false">   class MyConferenceManager : public CNmConferenceManager
   {
      public :
      
         // implement some or all of the INmManagerNotify methods to do stuff when the notifications happen...
   };
   
...
   
   if (MyConferenceManager::IsNmInstalled())
   {
      MyConferenceManager confManager;
   
      ULONG options = NM_INIT_NORMAL;
      ULONG capabilities = NMCH_ALL;
   
      confManager.Initialize(options, capabilities);
   
... </pre>


<p>Since the <code>CNmConferenceManager</code> object implements all of the notification interface and just does nothing, i.e. it returns <code>S_OK</code> to all methods, our derived class can choose to simply override the notifications that it wants to handle. Creating our object initialises COM, so that COM remains initialised until our object is no more, creates the NetMeeting Conference Manager COM object and connects up the notification sink. We can then simply call methods on the object and recieve notifications back.</p>

<p><b>Download</b><br />
The following source was built using Visual Studio 6.0 SP5 and Visual Studio .Net. You need to have the Microsoft NetMeeting SDK installed. This is available from <a href="http://www.microsoft.com/windows/netmeeting/authors/sdk/default.asp">here</a> or as part of recent versions of the <a href="http://www.microsoft.com/msdownload/platformsdk/sdkupdate/">Microsoft Platform SDK</a>.</p>

<p><a href="http://www.lenholgate.com/zips/COMNotify.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'COMNotify.zip']);">COMNotify.zip - an example notification sink using the NetMeeting Conference Manager COM object.</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>30th May 2002 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>.</li>
<li>20th September 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul><p></p>]]>
    </content>
</entry>

<entry>
    <title>Designing asynchonous COM components for VB</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2002/04/designing-asynchonous-com-components-for-vb.html" />
    <id>tag:www.socketframework.com,2002:/blog//12.141</id>

    <published>2002-04-11T00:00:00Z</published>
    <updated>2010-12-22T13:04:12Z</updated>

    <summary>This example COM component provides three COM objects for using the Win32 Mailslot IPC mechanism. The component may be useful if you need to communicate from VB using Mailslots. However, the reason I wrote it was to demonstrate creating a...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        This example COM component provides three COM objects for using the Win32 Mailslot IPC mechanism.  The component may be useful if you need to communicate from VB using Mailslots. However, the reason I wrote it was to demonstrate creating a COM component in C++ that integrates well with VB and can fire asynchronous events.
        <![CDATA[<p><b>Overview</b><br />
The COM component consists of an object factory which is used to create instances of the Mailslot manipulation objects. There are three Mailslot objects: a ClientMailsot object which provides the 'write' end of a Mailslot connection; a synchronous ServerMailslot object which provides the 'read' end of a Mailslot connection but which needs to be polled to receive data; and an asynchronous AsyncServerMailslot object which signals the arrival of data by firing an event. First we will take a look at the object model for the component as this reveals several tricks that make it easy to use the objects from within VB. Then we take a look at the implementation and address the threading issues that arise when a component can operate asynchronously.</p>

<p><b>Interface design: Object creation</b><br />
The factory object exists so that you can create and configure the mailslot objects as a one step process. COM objects cannot have the equivalent of C++ constructors so without a factory object you would have to create the mailslot object and then configure it. If you neglected to configure the object, or if you attempted to configure it but the configuration failed, you could end up with an object which exists within your program but is useless. It's far better to prevent the creation of these zombie objects by wrapping the creation and configuration of an object into a single step. If this succeeds then you have your object and it's correctly configured and operational, if it fails then you never get given an object.</p>

<p>The factory object's interface IDL looks something like this:</p>


<pre class="brush: cpp gutter: false">   interface IMailslotFactory : IDispatch
   {
      HRESULT CreateClientMailslot(
         [in] BSTR name, 
         [in, optional] VARIANT computerOrDomain, 
         [out, retval] IClientMailslot **ppSlot);
  
      HRESULT CreateServerMailslot(
         [in] BSTR name,
         [in, optional] VARIANT maxMessageSize,
         [in, optional] VARIANT readTimeOut,
         [out, retval] IServerMailslot **ppSlot);
  
      HRESULT CreateAsyncServerMailslot(
         [in] BSTR name, 
         [in, optional] VARIANT maxMessageSize, 
         [out, retval] IAsyncServerMailslot **ppSlot);
   };</pre>


<p>The factory object itself is marked as 'appobject' which means that these methods are available for use within VB without specifying an object reference explicitly, allowing code such as this to be written:</p>


<pre class="brush: cpp gutter: false">   Dim slot As JBCOMMAILSLOTLib.ClientMailslot
  
   Set slot = CreateClientMailslot("MySlot")
</pre>


<p>Each method creates and configures the corresponding mailslot object. If the configuration fails then no object is returned and an error is raised.</p>

<p>Internally the object factory works by creating an instance of the COM object required but requesting an 'initialisation' interface rather than the normal client facing interface. The initialisation interface doesn't need to be exposed in the IDL or type library as it's only for internal use within the component.</p>

<p>The initialisation interface for the ClientMailslot object is defined as follows:</p>


<pre class="brush: cpp gutter: false">   class __declspec(uuid("589E7114-50EE-4598-9140-92610D9BC20F")) IClientMailslotInit;
   
   class ATL_NO_VTABLE IClientMailslotInit : public IUnknown
   {
      public :
   
         STDMETHOD(Init)(
            /*[in]*/ BSTR name,
            /*[in]*/ VARIANT computerOrDomain) = 0;
   };</pre>


<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379686&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;bg1=ffffff&amp;f=ifr" style="width:120px;height:280px;" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>The object factory passes the user supplied parameters through to the ClientMailslot which can attempt to configure itself. Failure results in the factory destroying the ClientMailslot and returning the error to the caller.</p>

<p>If object initialisation succeeds the factory queries the ClientMailslot for its client facing interface, <code>IClientMailslot</code>, and releases the initialisation interface. The ClientMailslot is then returned to the caller as a completely initialised and operational object.</p>

<p>To prevent the user creating an object directly, rather than using the object factory, the other objects are marked as noncreatable in their IDL, also notice that the IDL doesn't mention the initialisation interface.</p>


<pre class="brush: cpp gutter: false">   [
      uuid(30A92485-94D2-4CBA-AC32-EF276B7F777B),
      helpstring("ClientMailslot Class"),
      noncreatable
   ]
   coclass ClientMailslot
   {
      [default] interface IClientMailslot;
   };</pre>


<p>The ServerMailslot and AsyncServerMailslot are created by the factory in the same manner.</p>

<p><b>Interface design: Data transmission</b><br />
Data can be sent either as a string or as an array of bytes. Likewise, data can be receieved in either format. Sending in one format does not prescribe how the server can receive the data.</p>

<p>The <span class="caps">IDL </span>for the ClientMailsot interface is something like this:</p>


<pre class="brush: cpp gutter: false">   interface IClientMailslot : IDispatch
   {
      HRESULT WriteString(
         [in] BSTR data);
  		
      HRESULT Write(
         [in] VARIANT arrayOfBytes);
   };</pre>


<p>This can be used as follows:</p>


<pre class="brush: cpp gutter: false">   Private Sub SendString_Click()
  
      m_slot.WriteString MessageEdit.Text
   
   End Sub
  
   Private Sub SendBytes_Click()
  
      Dim stringLength As Integer
      stringLength = Len(MessageEdit.Text)
  
      Dim bytes() As Byte
  
      ReDim bytes(stringLength)
  
      Dim i As Integer
  
      For i = 0 To stringLength - 1
         bytes(i) = Asc(Mid(MessageEdit.Text, i + 1, 1))
      Next i
  
      m_slot.Write bytes
  
   End Sub</pre>


<p>Note that by sending a string you are actually sending the Unicode string: by sending "AAAA" as a string you actually send the follwoing bytes: <code>0x65 0x00 0x65 0x00 0x65 0x00 0x65 0x00</code>.</p>

<p>The sychronous ServerMailsot provides corresponding read methods. When a message is available it can be read either as bytes or as a string. However, calling either of the read methods will consume the current message. You cannot call <code>Read()</code> to read a message as an array of bytes and then <code>ReadString()</code> to read the same message as a string, the call to <code>ReadString()</code> will attempt to read the next available message. If you attempt to read a message and there is not one available within the read timeout period then an error is raised. Because the read call is synchronous your code could block in the read call for the length of the read timeout period. The read timeout is specified when you create the ServerMailslot and not on a per read basis.</p>

<p>The asynchronous AsyncServerMailslot delivers messages when they arrive via an event. The IDL for the event interface looks something like this:</p>


<pre class="brush: cpp gutter: false">   dispinterface _IAsyncServerMailslotEvents
   {
      properties:
      methods:
   
         HRESULT OnDataRecieved(
            [in] IMailslotData *mailslotData);
   };</pre>


<p>and the event can be handled in VB like this:</p>


<pre class="brush: cpp gutter: false">   Private Sub m_slot_OnDataRecieved(ByVal mailslotData As JBCOMMAILSLOTLib.IMailslotData)
            
      Dim stringData as String
      stringData = mailslotData.ReadString()
   
      ' do something with the string...    
   
      Dim bytes() As Byte
      bytes = mailslotData.Read()
   
      ' do something with the bytes
   
   End Sub</pre>


<p>The MailslotData object encapsulates a single mailslot message and should not be retained outside scope of the event handler. If you want to keep the data, extract it as either a string or an array of bytes and keep that representation. This limitation is due to how the AsyncMailslotServer object optimises the event dispatch mechanism - only one MailslotData object exists per AsyncMailslotServer and it is reused for each message that arrives.</p>

<p>Unlike the ServerMailslot you can call both <code>ReadString()</code> and <code>Read()</code> on the MailslotData object to receieve the same message data in either format.</p>

<p><b>Implementation issues: The CCOMMailslot helper object</b><br />
Implementation of the ClientMailslot object is pretty simple. It offers only two write methods and the internal initialisation method. All of the actual work is deferred to a helper object, <code>CCOMMailslot</code>, which deals with the code that's common between all of the Mailslot COM objects. The only work that the ClientMailslot actually does is to extract the data from the supplied byte array.</p>

<p>The ServerMailslot is equally straight forward. Most of the methods are implemented by <code>CCOMMailslot</code> with only the <code>Read</code> methods requiring any work within the ServerMailslot object itself. Both <code>Read()</code> and <code>ReadString()</code> call down to <code>CCOMMailslot::Read()</code> and then package the resulting data as either a <code>BSTR</code> or a SafeArray of bytes.</p>

<p>The majority of the work that <code>CCOMMailslot</code> does is simply parameter checking and the wrapping of the Win32 Mailslot API. The only slightly complex code is to be found in the read method. Mailslots can be created with a maximum message size, in which case we know the size of the buffer required for read operations, they can also be created with an unspecified message size which accepts messages of any size. If the Mailslot was created with a maximum message size then we simply allocate a buffer large enough and use that for each read. If there was no maximum specified then we first call <code>SizeOfWaitingMessage()</code> to see if there is a message waiting and if so to retrieve the size of the message. If there is a message waiting we expand the size of our buffer, if necessary, so that we have enough space to read the message, we then read the message in.</p>

<p><b>Implementation issues: The CAsyncCOMMailslot helper object</b><br />
Not surprisingly the most complex object is the aysnchronous AsyncServerMailslot. This object is multi-threaded and generates events when messages arrive on the Mailslot. It's generally considered unwise[<a href="http://www.amazon.co.uk/exec/obidos/redirect?path=ASIN/0201379686&amp;link_code=as2&amp;camp=1634&amp;tag=ramcom-21&amp;creative=6738">1</a><img src="http://www.assoc-amazon.co.uk/e/ir?t=ramcom-21&amp;l=as2&amp;o=2&amp;a=0201379686" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />] to create multi-threaded DLL hosted COM components. However, the threads created in this component are tied to the lifetime of the AsyncServerMailslot objects so we can guarantee that all worker threads will have ceased by the time the component is to be unloaded. If you're concerned about this then the code could easilly be housed in an EXE component. I feel that the convenience of having a single dll component which includes all required proxy/stub code is worth it in this situation.</p>

<p>When an AsyncServerMailslot is created it spawns a worker thread which performs infinitely blocking, overlapped reads on the Mailslot handle. The worker thread blocks waiting for either the read to complete or for its shutdown event to be signalled. When a read completes the receieved data is wrapped in a MailslotData object and the event is fired to alert clients.</p>

<p>Due to the "Rules of COM" the event sink must either be fired from the same thread that was used to register it or the event sink interface must be marshalled to the thread that will fire the event. Due to how ATL generates Connection Point code for us it's not practical to marshal each event sink to a worker thread to fire the event so instead we opt to fire the event from the component's main thread. To fire the event from the component's main thread we need to have the worker thread communicate with the main thread, one method of doing this is to use window messages another is to marshal an interface from the main thread to the worker thread. The window message method is explained in one of Microsoft's knowledge base articles[<a href="http://support.microsoft.com/directory/article.asp?ID=KB;EN-US;Q196026&amp;">2</a>] but requires us to create a dummy window and add other clutter to our code, the interface marshalling method is slightly more complex but offers us some advantages.</p>

<p>When an object needs to operate in a multi-threaded way and needs to call back into itself via COM it should marshal an interface to the worker thread using <code>CoMarshalInterThreadInterfaceInStream()</code>. Inside the worker thread the interface is unmarshalled using <code>CoGetInterfaceAndReleaseStream()</code> and can be used to safely communicate with the component's main thread via COM. This works fine until the time comes to unload the component. Unfortunately by marshalling the interface within the component you have created a circular reference cycle. The component is, essentially, holding a reference on itself and this outstanding reference will prevent the component being destroyed. Since the internal reference will be held until the worker thread shuts down and the worker thread only shuts down when the component is destroyed you can see we have a problem.</p>

<p><b>Implementation issues: Reference cycles and weak identities</b><br />
The circular reference cycle problem is generally solved using the "split identity" or "weak reference" idioms[<a href="http://www.amazon.co.uk/exec/obidos/redirect?path=ASIN/1861002254&amp;link_code=as2&amp;camp=1634&amp;tag=ramcom-21&amp;creative=6738">3</a><img src="http://www.assoc-amazon.co.uk/e/ir?t=ramcom-21&amp;l=as2&amp;o=2&amp;a=1861002254" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />]. The idea is that the reference cycle is broken by a reference that does not affect the object's reference count or the object exposes a second identity (COM object) that, whilst part of the main object, doesnt affect the main object's reference count. Both of these techniques allow the main object to begin shutdown when all external references have been released. Weak references are fairly complex to achieve within <span class="caps">ATL </span>and although a solution is presented in [<a href="http://www.amazon.co.uk/exec/obidos/redirect?path=ASIN/1861002254&amp;link_code=as2&amp;camp=1634&amp;tag=ramcom-21&amp;creative=6738">3</a><img src="http://www.assoc-amazon.co.uk/e/ir?t=ramcom-21&amp;l=as2&amp;o=2&amp;a=1861002254" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />] it's overly complex and invasive for what we need here.</p>

<p>Our solution to the reference cycle generated by the interface that we're holding in the worker thread is to create a weak identity for the AsyncServerMailslot. This weak identity supports the interface that is required to communicate between the worker thread and the component's main thread. The weak identity is a simple COM object in its own right, it has its own implementation of <code>AddRef()</code>, <code>Release()</code> and <code>QueryInterface()</code> and, since the <code>IUnknown</code> interface returned is different from the main object it has its own identity in COM.</p>

<p>The IDL for the interface that we use for communicating between threads looks like this:</p>


<pre class="brush: cpp gutter: false">   interface _AsyncServerEvent : IUnknown
   {
      HRESULT OnDataRecieved();
   };</pre>


<p>Essentially it's just a way for the worker thread to "prod" the main thread. This interface appears in the IDL file because although we only use the interface internally and none of the publically visable objects expose the object we need to have proxy/stub code generated for it.</p>

<p>The weak identity we need looks like this:</p>


<pre class="brush: cpp gutter: false">   class CAsyncServerEventHelper : public _AsyncServerEvent
   {
      public :
  
         CAsyncServerEventHelper(_AsyncServerEvent &amp;theInterface);
  
         STDMETHOD(OnDataRecieved)();
  
         // IUnknown methods
         ULONG STDMETHODCALLTYPE AddRef();
         ULONG STDMETHODCALLTYPE Release();
         STDMETHOD(QueryInterface(REFIID riid, PVOID *ppvObj));
   
      private :
   
         _AsyncServerEvent &amp;m_interface;
   };</pre>


<p>and is implemented like this:</p>


<pre class="brush: cpp gutter: false">   CAsyncServerEventHelper::CAsyncServerEventHelper(_AsyncServerEvent &amp;theInterfce)
      :  m_interface(theInterfce)
   {
   } 
   
   STDMETHODIMP CAsyncServerEventHelper::OnDataRecieved()
   {
      return m_interface.OnDataRecieved();
   }
   
   ULONG STDMETHODCALLTYPE CAsyncServerEventHelper::AddRef()
   {
      return 2;
   }
   
   ULONG STDMETHODCALLTYPE CAsyncServerEventHelper::Release()
   {
      return 1;
   }
  
   STDMETHODIMP CAsyncServerEventHelper::QueryInterface(REFIID riid, PVOID *ppvObj)
   {
      if (riid == IID_IUnknown || riid == IID__AsyncServerEvent)
      {
         *ppvObj = this;
         AddRef();
         return S_OK;
      }
     
      return E_NOINTERFACE;
   }</pre>


<p>Our main COM identity has a member variable of type <code>CAsyncServerEventHelper</code> which it initialises with a pointer to itself (this) in its constructor. It then marshals the weak identity's <code>_AsyncServerEvent</code> interface to its worker thread, this creates the appropriate proxy so that calls to the interface are marshalled across threads correctly, but it doesn't affect the reference count of the main identity.</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=1861002254&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;bg1=ffffff&amp;f=ifr" style="width:120px;height:280px;" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>When the worker thread reads data from the Mailslot it calls the <code>OnDataRecieved()</code> method of the interface that it unmarshalled, this causes the call to be marshalled into the component's main thread where the helper object passes the call on to the main object. In this example we don't bother to pass any data across in the call, we just use it as a way of having one thread "poke" another. The worker thread reads data into the read buffer and pokes the main thread, the main thread then uses the data that has just been read and fires the events in all connected clients. At first this may look dangerous, but we're using the sychronous nature of the STA apartment of our main object to provide synchronisation across the call. The worker thread will block until the main thread completes the event dispatch.</p>

<p>Of course this is but one way of implementing a weak identity, but it's relatively simple, unobtrusive and works well.</p>

<p><b>Conclusions</b><br />
Designing COM components that integrate well with VB is fairly straight forward if you follow some simple rules when desiging your interfaces.</p>

<p>Working with asycnhronous events is easy if you follow the rules of COM, are aware of when you're creating reference cycles and know how to break them.</p>

<p><b>References</b></p>

<ul>
<li>[1] <a href="http://www.amazon.co.uk/exec/obidos/redirect?path=ASIN/0201379686&amp;link_code=as2&amp;camp=1634&amp;tag=ramcom-21&amp;creative=6738">Effective COM: 50 Ways to Improve Your COM and MTS-based Applications (Object Technology S.)</a><img src="http://www.assoc-amazon.co.uk/e/ir?t=ramcom-21&amp;l=as2&amp;o=2&amp;a=0201379686" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> - Item 32</li>
<li>[2] <a href="http://support.microsoft.com/directory/article.asp?ID=KB;EN-US;Q196026&amp;">Q196026</a> - PRB: Firing Event in Second Thread Causes IPF or GPF</li>
<li>[3] <a href="http://www.wrox.com/ACON11.asp?WROXEMPTOKEN=760156ZsaY74TVaHd37Pvzqn7r&amp;ISBN=1861002254">COM IDL &amp; Interface Design</a> - Chapter 5</li>
</ul>

<p><b>Download</b><br />
The following source built using Visual Studio 6.0 SP5. Using the November 2001 edition of the Platform SDK.<br />
<a href="http://www.lenholgate.com/zips/JBCOMMailslot.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JBCOMMailslot.zip']);">JBCOMMailslot.zip - C++ COM component source code and VB examples</a></p>

<p><b>Revision History</b></p>
<ul>
<li>11th April 2002 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>.</li>
<li>2nd September 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul>]]>
    </content>
</entry>

<entry>
    <title>AVL Tree</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/08/avl-tree.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.140</id>

    <published>2001-08-07T10:31:56Z</published>
    <updated>2010-12-23T09:46:12Z</updated>

    <summary>A &quot;generic&quot; AVL Tree, from the dark days before templates... The code here is some of my first C++. Back in 1991 C++ was still pretty new. Looking back at my early C++ is better than looking back at my...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        A &quot;generic&quot; AVL Tree, from the dark days before templates... The code here is some of my first C++. Back in 1991 C++ was still pretty new. Looking back at my early C++ is better than looking back at my early C. At least my early C++ just looked like OKish C with some odd keywords...
        <![CDATA[<p><b>How do you index a data structure that keeps changing?</b><br />
The game world for the multi-user adventure system I was writing between '89 and '94 was stored in a network of interconnected nodes. I called it a Semantic network because of some AI book I'd read at the time, the whole of the knowledge of the game world was in this network. The design of the game meant that people could create new objects or rooms via magical powers, alternatively they completely remove items from the world. Because of this dynamic nature of the world itself the index into the world needed to be able to handle regular inserts and deletes without degenerating... I remember reading about AVL trees in the Turner Simms concert hall at Southampton University whilst watching my girlfriend rehearse (<a href="http://www.amazon.co.uk/exec/obidos/redirect?path=ASIN/0201072564&amp;link_code=as2&amp;camp=1634&amp;tag=ramcom-21&amp;creative=6738">Data Structure Techniques</a><img src="http://www.assoc-amazon.co.uk/e/ir?t=ramcom-21&amp;l=as2&amp;o=2&amp;a=0201072564" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> - Standish - 0-201-07256-4 page 108...). I decided that they sounded like the ideal index into my network. I planned to have some form of trie index at the top level with AVL trees below each letter...</p>

<p><b>What's an AVL tree then?</b><br />
<a href="http://www.google.com/custom?domains=lenholgate.com&amp;q=AVL+Tree&amp;sitesearch=&amp;client=pub-6888952347469638&amp;forid=1&amp;ie=ISO-8859-1&amp;oe=ISO-8859-1&amp;safe=active&amp;cof=GALT%3A%23008000%3BGL%3A1%3BDIV%3A%23336699%3BVLC%3A663399%3BAH%3Acenter%3BBGC%3AFFFFFF%3BLBGC%3A336699%3BALC%3A0000FF%3BLC%3A0000FF%3BT%3A000000%3BGFNT%3A0000FF%3BGIMP%3A0000FF%3BFORID%3A1%3B&amp;hl=en">AVL</a> trees, named after the Russians, Adel'son-Vel'skii and Landis, who first defined them, are height balanced binary search trees. To be considered an AVL tree a binary search tree must satisfy only one condition: for any node, the height of the left sub-tree and the height of the right sub-tree must differ by no more than one. As nodes are added and removed one or more node may lose the AVL property and need to be re-balanced. This can be achieved by applying one of four "rotations", this is far more efficient than having to reorganise the whole tree as you would to keep a binary tree perfectly balanced. In the worst case, a search for a specific node in an AVL tree will require 45% more comparisons than a perfectly balanced binary search tree. Whilst in the average case the additional comparisons required are negligable.</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=6&amp;l=as1&amp;asins=0201072564&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;f=ifr&amp;bg1=ffffff&amp;noImg=1" width="120" height="210" scrolling="no" marginwidth="0" marginheight="0" frameborder="0">&amp;amp;amp;lt;br /&amp;amp;amp;gt;
</iframe>
<b>The four rotations...</b><br />
In all of the diagrams below the node "A" is being examined for the AVL property and requires a rotation to restore it. Each rotation will return node "B" which will have been adjusted so that it meets the requirements of the AVL tree.</p>

<p><i>Rotate left</i></p>

<p align="center"><img alt="AVL tree - rotate left" src="http://www.lenholgate.com/blog/images/RotateLeft.gif" border="0" /></p>

<p><i>Double rotate left</i></p>

<p align="center"><img alt="AVL tree - double rotate left" src="http://www.lenholgate.com/blog/images/DoubleLeft.gif" border="0" /></p>

<p><i>Rotate right</i></p>

<p align="center"><img alt="AVL tree - rotate right" src="http://www.lenholgate.com/blog/images/RotateRight.gif" border="0" /></p>

<p><i>Double rotate right</i></p>

<p align="center"><img alt="AVL tree - double rotate right" src="http://www.lenholgate.com/blog/images/DoubleRight.gif" border="0" /></p>

<p>Additions of nodes deep in the tree can cause a cascading series of rotations all the way back to the root. This can be clearly seen from the test harness.</p>

<p><b>An ideal "learning C++" project... (pity I didn't really use C++)</b><br />
Since I was learning C++ at the time I decided to implement the index in C++ (I later converted it to C for the Unix port of the game...). Having written it I wanted to be able to store all manner of things in it, so I made it "generic". At the time the best I could do was tell the tree how big the items you were storing were and supply a comparison function. The compiler of the day didn't support templates but why my generic datatype of choice was a <code>BYTE *</code> rather than a <code>void *</code> I don't know. What scares me now is looking at the code and seeing how huge the member functions are and how much is going on. There's no use of abstraction to make things easier to understand, nothing's broken into easily digested chunks. There's lots of comments though, so it must be good ;). Also I really disliked those C++ stream things, lets stick with <code>printf()</code> it's nicer...</p>

<p>I can't remember why I use the complicated stack based method of traversing the tree. I know I wrote a version which used recursion but I think I managed to blow the stack when testing it on DOS and so switched to a method that wouldn't blow the stack...</p>

<p>I'd just read about using gotos to get out of horrible error situations and was trying it out. Though I believe the technique of using multiple gotos to jump to a single clean up point on error is OK I wouldn't use it often today.</p>

<p>I remember playing for hours with this looking for a bug that I don't recall ever finding (can't think why, when the code's so easy to read...) I seem to recall that the tree became unlinked once or twice and lost some nodes...</p>

<p>The test program adds letters to a tree and displays the tree as it dynamically balances. It's cute to watch.</p>

<p>I might sit down and write a template version that uses recursion rather than the stack climbing method employed here. We'll see.</p>

<p><b>Faster than it used to be...</b><br />
Having just downloaded and run this code for the first time in forever I notice that it runs a little too fast on modern hardware. Adding the following slows it down to something that you can see...</p>

<p>In the code that looks like this:<br />
</p><pre class="brush: cpp gutter: false">#include &lt; math.h &gt;   // pow
#include &lt; dos.h &gt;     // sleep
#include &lt; time.h &gt;    // time
  
int count;
  
int compChar(BYTE *itemOne, BYTE *itemTwo)</pre>
Add this:
<pre class="brush: cpp gutter: false">#include &lt; math.h &gt;   // pow
#include &lt; dos.h &gt;     // sleep
#include &lt; time.h &gt;    // time
  
#include &lt; Windows.h &gt; // &lt;----
  
int count;
    
int compChar(BYTE *itemOne, BYTE *itemTwo)</pre>
And in the code that looks like this:
<pre class="brush: cpp gutter: false">      //count++;
      if (count == 10) {
         fprintf(stdout,"\n%s\n",dispBufPtr);
         count = 0;
      }
      printf("\n%s\n",dispBufPtr);
  
      delete dispBufPtr;
}
#endif</pre>
<p>Add this:</p>
<pre class="brush: cpp gutter: false">      //count++;
      if (count == 10) {
         fprintf(stdout,"\n%s\n",dispBufPtr);
         count = 0;
      }
      printf("\n%s\n",dispBufPtr);
  
      delete dispBufPtr;
      ::Sleep(500);       // &lt;----
}
#endif</pre>
<p><b>Download</b><br />
The source was originally built with Borland C++ v3.1 but it compiled up straight away on VC++ v5.0.</p>

<p><a href="http://www.lenholgate.com/zips/GAVL.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'GAVL.zip']);">GAVL.zip</a></p>]]>
    </content>
</entry>

<entry>
    <title>Rooms 1.7</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/08/rooms-17.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.139</id>

    <published>2001-08-02T10:25:01Z</published>
    <updated>2010-12-22T13:08:03Z</updated>

    <summary>In 1989 I taught myself C to write an adventure game, the code&apos;s fairly bad, but 10 year&apos;s later, the game&apos;s still quite cool!...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        In 1989 I taught myself C to write an adventure game, the code&apos;s fairly bad, but 10 year&apos;s later, the game&apos;s still quite cool!
        <![CDATA[<p><b>A very simple adventure game...</b><br />
Over 10 years ago I taught myself to program in C. I wanted to write a program to help me customise the multi-user adventure game I was playing at the time.  In <a href="http://www.mud.co.uk/richard/vax.htm">MUD II</a> when you reached the exalted rank of Wizard you could create your own rooms and objects. To do this you had to execute lots and lots of commands within the game to create the objects etc. To make this repeatable it was common to script the creation of these objects. What I wanted to write was a room and object editor that allowed me to create these  scripts offline and in a more user-friendly way.</p>

<p>Well, that's how the "Rooms" project started. It never did generate any scripts and after a while I decided that it might be fun to write my own multi-user game and "Rooms" became ELGAR - Endless Lands Games And Realities, my multi-user adventure game language project.</p>

<p><a href="http://www.lenholgate.com/blog/assets_c/2010/12/Rooms17-247.html" onclick="window.open('http://www.lenholgate.com/blog/assets_c/2010/12/Rooms17-247.html','popup','width=652,height=331,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false"><img src="http://www.lenholgate.com/blog/assets_c/2010/12/Rooms17-thumb-500x253-247.gif" width="500" height="253" alt="Rooms17.gif" class="mt-image-none" style="" /></a></p>

<p>The code presented here is the last version of the Rooms source code. This was version 1.7 and by this point most of the pretence of just writing a room editor for <a href="http://www.mud.co.uk/richard/rhapr92.htm">MUD II</a> had been dropped and I was already experimenting with adventure game ideas.</p>

<p>The code is rather horrible, but, I was young, and it was my first C program, albeit version 1.7 of it...</p>

<p>The data for the "game" are stored in the various MN.* files, you can alter these with a text editor though there is a limit to the number of rooms, objects and messages that you can have. I did know the format for these files at one time, but can't for the life of me remember that now... I'm too scared to look into the code to find out.</p>

<p><b>Things I'm still quite proud of...</b><br />
Objects and "features" are "wrapped" into the room descriptions. This was pretty much unheard of at the time. Small objects often appear early on in the room description and don't show in up at all if you do a "quick look" (ql) but you can still pick them up. Also, most things mentioned in the room descriptions can be touched etc e.g. the ladder, stairs, racks, forge etc. You can open/close and put things in the forge even though it looks like it's part of the room description. This was all very unusual for this kind of game back in 1989.</p>

<p>Try Examining things, the scabbard and Longsword (ls) are nice, take the ls from the scabbard and examine it whilst holding it, then drop it and examine it. Find the chest and examine it when shut, when held, and when open... Then try the safe... Things like drop 1 gold coin, and put fifteen ingots in the oak chest work, as I remember, but the parser can be easily fooled, try playing with the war axe, I think you can call it all kinds of things "bladed large handled axe" etc most of which aren't supposed to work...</p>

<p>There's a where spell, oh, the trap-door in the eastern end of the cellar is nice. Check the description when closed and again when open, go outside and close it and it's "hidden". You're supposed to be unable to open it from that side, but I never got around to that.</p>

<p>Things like the ingots, coins etc have different descriptions for single and multiple instances of them, try g iron ingot, dr 1 ingot, ql,dr 1 ingot,ql</p>

<p>Commands can be strung together n,n,n,n etc you can't repeat a command with a single period though (n....), then again, MUD II didn't do that at the time of my writing Rooms...</p>

<p>As you can see from the commands and the SCore it was slowly turning into the dry run for a multi-player thing. Try out the various types of personae, I never did get around to setting an "opening size" for the doors, so the giant can go anywhere and carry everything - except  features - and the fairy can carry very little - a tarot card only, I think...</p>

<p>Anyway, download, unzip, run the batch file to run the game and don't be too harsh on the lack of quality of the C code...</p>

<p><b>Download</b><br />
The source for Rooms v1.7 was for DOS and built with Borland C++ v3.1. The code is presented "as is" in all its "my first C program" glory. The compiled executable is included just in case...</p>

<p><a href="http://www.lenholgate.com/zips/Rooms17.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Rooms17.zip']);">Rooms17.zip</a></p>]]>
    </content>
</entry>

<entry>
    <title>Java caches in the middle tier</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/07/java-caches-in-the-middle-tier.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.138</id>

    <published>2001-07-26T19:18:27Z</published>
    <updated>2010-12-28T13:33:43Z</updated>

    <summary>A common way to improve the performance of Java code is to cache objects rather than repeatedly create and destroy them. This is especially true when you&apos;re writing middle tier servers that service client requests and return results objects. Implementing a flexible caching scheme in Java is relatively easy, but there are a few things to watch for.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Java" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        A common way to improve the performance of Java code is to cache objects rather than repeatedly create and destroy them. This is especially true when you&apos;re writing middle tier servers that service client requests and return results objects. Implementing a flexible caching scheme in Java is relatively easy, but there are a few things to watch for.
        <![CDATA[<p><b>The problem</b><br />
Creating objects in the middle tier that live for only the length of a single client request can be inefficient. When those objects are the results from  client requests, and when serveral clients may make requests for the same data, it often makes sense to cache the results. Caching in Java can be implemented relatively easily. After all, the simplest cache is just a map which can store objects based on keys. Sometimes this simple solution is good enough, but usually you want more control over the lifetime of the objects that live in the cache.</p>

<p>This article uses a simple CORBA server to demonstrate increasingly complex cache implementations. This server implements the following interface which allows a client to pass a string and receive a <code>String[][]</code> in return. As we go along we'll improve the caches that it uses and refactor the code as our design evolves.</p>


<pre class="brush: cpp gutter: false">typedef sequence&lt;string&gt; StringSeq;
typedef sequence&lt;StringSeq&gt; StringSeqSeq;

interface CacheTest
{
   StringSeqSeq DoSelect(in string sql);
}
</pre>

<p>In <a href="http://www.lenholgate.com/zips/JCache1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache1.zip']);">JCache1.zip</a> we develop the client/server framework and use a HashMap as a simple cache. We simulate a time-intensive operation to fetch data based on the user's request and store the results of the queries in the cache for later reuse. The following code from CacheTestImple.java does the work:</p>


<pre class="brush: cpp gutter: false">   public String[][] DoSelect(String sql)
   {
      String[][] results = findInCache(sql);

      if (results == null)
      {
         results = getData(sql);
   
         addToCache(sql, results);
      }

      return results;
   }

   private String[][] getData(String sql)
   { 
      System.out.println("getData(\"" + sql + "\")");

      // replace this with a call to a database if you want...

      pause(5000);

      String[][] results = new String[2][2];

      results[0][0] = "Timestamp";
      results[0][1] = "Sql";
      results[1][0] = new Date().toString();
      results[1][1] = sql;
     
      return results;
   }
</pre>

<p>Obviously this is a simplified example, but the concept is valid. If data being retrieved by a request takes some time to obtain, and multiple clients are likely to request it, then it makes sense to cache it. What's more, due to the way that Java often garbage collects, you may find that a server that caches every result actually has a smaller memory footprint than one that doesn't!</p>

<p><b>Building the example code</b><br />
Install Ant, the build system used by JacORB, and type 'ant' in the src directory. If you don't want to install Ant then it's fairly easy to take the build.xml files and convert them into normal makefiles.</p>

<p><b>Running the sample</b><br />
The scripts directory contains a script to run the server and a script to run the client. The client/server pair use the CORBA naming service to find each other. Run up the JacORB name server (you can use the ns.bat file and the supplied orb.properties and jacorb.properties if you want to). Then run up the server and run the client. By passing different strings on the command line you can make the server return different 'result objects' when it needs to create a new result object there is a delay, when it doesn't because it found the results in the cache the server returns quicker.</p>

<p><b>A performance tweak for the key to the map</b><br />
Although caching the results in a simple HashMap improves the performance of our server there's a small performance tweak that's probably worth making at this stage. Strings aren't the best choice for keys in a HashMap as they calculate their hash value every time the hashCode() member function is called. A simple CacheKey can wrap a String and cache the hash... In <a href="http://www.lenholgate.com/zips/JCache2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache2.zip']);">JCache2.zip</a> we add a CacheKey class which does just that.</p>

<p><b>Improving on a map</b><br />
A more complicated cache will need to be able to have some control over the entries that it contains. At present our cache will simply grow and grow as new entries are added to it. However, this may be appropriate for some servers and as we've decided that the caching strategy required  by one server may not be the same as that required by another server, or even by another cache within the same server, we'll create an interface that all caches can implement. We will then be able to plug new caches in with ease as long as they conform to our interface.</p>

<p>Our cache interface can look something like this, and we'll move the CacheKey class inside the interface as it's so closely related. Since the second Cache in Cache.CacheKey is a bit redundant, we'll lose it.</p>


<pre class="brush: cpp gutter: false">public interface Cache
{
   public Object find(String keyString);
   public Object find(Key cacheKey);

   public void add(String keyValue, Object obj);
   public void add(Key cacheKey, Object obj);
   
   public void remove(String keyValue);
   public void remove(Key cacheKey);
   
   public String name();
   
   public int entries();
   public int hits();
   public int misses();
   
   public void flush();

   public class Key implements Comparable
   {
      public Key()
...
</pre>

<p>Next we take our original simple HashMap based cache and implement a SimpleCache which operates in the same way as our "forever growing" cache. We then adjust our server implementation class to use our cache interface and we're done. <a href="http://www.lenholgate.com/zips/JCache3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache3.zip']);">JCache3.zip</a> hasn't changed how our server caches its results, but it has made it far easier for us to change it in the future.</p>

<p><b>A slightly smarter cache</b><br />
If the server is going to run for long periods of time then a cache that never purges any entries is unlikely to work that well, unless the possible data set that the server can return is very small. A more useful cache might purge entries that haven't been used for some time so that once a result has been requested subsequent requests that occur within a certain time of each other will work from the cache. If the object isn't requested for some time, however, it will be purged to make room for other more commonly requested objects.</p>

<p>To achieve this we need to associate a last used timestamp with each cache entry. We then need some way to store the cache entries in descending order of when they were last used - so we can quickly walk through the entries from least recently used to most recently used. We also need a seperate thread to periodically scan the list to see if the least recently use object in the cache has timed out.</p>

<p>In <a href="http://www.lenholgate.com/zips/JCache4.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache4.zip']);">JCache4.zip</a> we implement a second cache that conforms to our Cache interface. TimeLimitedCache contains a HashMap which we use as the cache and a LinkedList which we use to store the cache elements in least recently used order. We add an ObjectRef class to the Cache interface. The ObjectRef links the object that we're caching with a timestamp and its key. This makes it easy for the entries to be removed when they time out. Finally we have a thread to periodically check the cache for objects that have timed out. The whole thing is controlled by two member variables, scanEveryMillis - how often the cleanup thread looks for timeouts - and entryTimeoutMillis - how long an entry can stay in the cache after the last client touched it. Both of these variables are set to very low values so that we can see the cache in action.</p>

<p>We've changed the client too, it now runs automatically. First it populates the server cache with 10 items and then it requests just one item repeatedly to show that a regularly requested item will remain in the cache after items that aren't being used are removed. The result objects that the server returns have been adjusted so that they are larger and the server run script now runs the Java VM with verbose garbage collection so that we can see the effect of the cache on the server's memory usage.</p>

<p><b>Taking out the trash</b><br />
If you watch the latest server carefully you'll notice that it doesn't garbage collect the objects that have been removed from the cache until an arbitrary time after they've been removed. Between the time that the objects are removed from the cache and the time that they are garbage collected the objects are still in memory but completely unusable.</p>

<p>This seems like a bit of a waste. In V1.2 of the Java SDK some classes for working with the garbage collector were added. One of the most useful was the SoftReference class. The SoftReference is a class that can refer to another Object, what makes the SoftReference a little special, however, is that if an object is only reachable through "Soft" references then it is eligible for garbage collection. You can access the object that a SoftReference refers to using the get() method. If the referent has been garbage collected then get() will return null. What's more, a SoftReference can be registered with a ReferenceQueue and at some point after the SoftReference's referent is garbage collected the SoftReference will be placed in the ReferenceQueue.</p>

<p>What this means is that you can use SoftReferences and ReferenceQueues to develop some fairly smart caches. By deriving our ObjectRef from a SoftReference and having the SoftReference hold a "soft" reference to our cache item and the ObjectRef hold an additional normal (hard) reference to the cache item we can release the normal reference when the object times out in the cache but still access the object right up until the moment that the garbage collector decides to collect it.</p>

<p>When the object in the cache times out, the cache calls releaseObject() on the ObjectRef. This causes the only "hard" reference to the object to be removed. The object is now only reachable by the "soft" reference and is therefore eligible to be garbage collected. The ObjectRef is still stored in the cache. If a request for the object comes in and the object hasn't yet been garbage collected then the ObjectRef will be located in the cache. The usage time is updated and get() is called to retrieve the object from the SoftReference. If the object still hasn't been collected then get() returns a reference to it, the hard reference is reattached, the user gets the object from the cache...</p>

<p>If, however, the object has been garbage collected then get() would return null and the cache lookup would have failed. Eventually the ObjectRef will be placed in the ReferenceQueue that it was associated with when it was constructed. By creating another thread we can poll the ReferenceQueue and remove ObjectRefs as they arrive after the object that they referred to has been collected. As we remove the ObjectRef from the queue we ask it to remove itself from the cache and at that point the purged object is no longer reachable via the cache...</p>

<p>It sounds more complicated than it is, go and look at the <a href="http://www.lenholgate.com/zips/JCache5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache5.zip']);">code</a>...</p>

<p><img src="http://www.lenholgate.com/blog/images/softref1.gif" /></p> 

<p>The ObjectRef holds a normal reference to the cached object so it won't be considered for garbage collection.</p>

<p><img src="http://www.lenholgate.com/blog/images/softref2.gif" /></p> 

<p>The hard reference is removed as the object is purged from the cache. Now the cached object can only be reached via a soft reference. It will now be considered for garbage collection.</p>

<p><img src="http://www.lenholgate.com/blog/images/softref3.gif" /></p> 

<p>The cached object is garbage collected. The ObjectRef is placed on the ReferenceQueue that was associated with the SoftReference that it derives from. When the ObjectRef is removed from the queue it can be removed from the cache.</p>

<p>Now when we run the client in <a href="/zips/JCache5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache5.zip']);">JCache5.zip</a> you'll see similar behaviour to before, but you'll also see each entry garbage collected. When there's only one entry left it will be allowed to timeout but then accessed before its garbage collected (if my timing code is right :)) and finally allowed to timeout and be garbage collected.</p>

<p><b>A time to live</b><br />
Some results objects may only be valid for a certain length of time. Perhaps you have data fed regularly into your database, say, once every hour. It's OK to cache client requests for up to an hour, but no longer. It doesn't matter if clients are requesting the same data every minute or two, after an hour you need to query the database again. The data has a maximum time to live and we should be able to create a cache which respects that time to live.</p>

<p>This new kind of cache has quite a lot in common with our TimeLimitedCache. It needs to hold an ObjectRef to the object and store a timestamp and it needs to periodically remove expired entries. If we separate this code out from our existing TimeLimitedCache we can produce a common base class for both the newly named EntryUsageTimeoutCache and our new MaxTimeToLivecache. The common base class will keep the TimeLimitedCache name. </p>

<p>EntryUsageTimeoutCache is the TimeLimitedCache from our previous example refactored so that the new TimeLimitedCache contains code that is common to both types of time limited caches. It provides the periodic cleanup thread and an interface that other time limited caches must adhere to for the cleanup thread to be able to work with them.</p>

<p>The ObjectRefs used by the two caches aren't actually as similar as they first seem. One needs a timestamp update whenever the object is accessed and the other needs a timestamp that is set only once, when the object is first added to the cache. To add both timestamps to Cache.ObjectRef seems inappropriate. So we'll move them into classes that derive from Cache.ObjectRef. As it happens, the ObjectRef used by MaxTimeToLiveCache doesnt want the magical object resurrection properties of the SoftReference based Cache.ObjectRef, but the SoftReference stuff is more likely to be used again so we'll leave that as the standard ObjectRef and just cruft up a new simpler one for the MaxTimeToLiveCache.</p>

<p>In <a href="http://www.lenholgate.com/zips/JCache6.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache6.zip']);">JCache6.zip</a> the server has been changed to use the new cache and the client remains the same. Notice how all of the cache entries are purged, even the one that's in regular use.</p>

<p><b>Endless caching strategies</b><br />
At this point we could continue creating new cache strategies until the cows come home... They're easy to use because of the common interface and we can write a cache to suit any server's requirements. The problem is that each time we decide that we need to change how we cache some data we need to modify the program to use the new cache. This is unfortunate and it means that we can't distribute our application with a "pretty good" cache and also ship the definition of the interface that users can use to write their own better caches if they are so inclined. Luckily, thanks to the wonders of reflection doing this is actually quite easy!</p>

<p>In <a href="http://www.lenholgate.com/zips/JCache7.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache7.zip']);">JCache7.zip</a> we add a CacheManager. The CacheManager manages our caches. It allows users of our caches to always work in terms of just the Cache interface. It's a factory for caches which can be configured by merging some properties with the system properties and then requesting a cache by name. The property file stores the mapping between the name that's hard coded in the source code and the cache that actually gets used. The CacheManager is a pluggable factory because it loads the required cache implementations dynamically from their .class files at run-time. It's user extensible because anyone can write a cache that conforms to our Cache interface, put the .class file in their class path and change the property file to refer to it, all without recompiling the application! That's pretty cool! The property file contains entries like this:</p>


<pre class="brush: text gutter: false">com.jetbyte.cache7.CacheManager.MYCACHE.CacheClass=com.jetbyte.cache7.EntryUsageTimeoutCache</pre>


<p>Where "MYCACHE" is the name of the cache passed to the CacheManager's getInstance() method. Usually you'd use the name of the object that contains the cache, such as com.jetbyte7.CacheTestImpl in the example. If the object has a need for more than one cache then it can append something to the end of its class name. If the object wishes to share a cache with another object, it can, as long as the two objects agree to call the cache by the same name. Run the new server and experiment by uncommenting each of the config lines in server.properties and restarting the server.</p>

<p><b>No caches</b><br />
Since we can now easily change the type of cache used, it would be nice to be able to turn off caching completely as conveniently. In <a href="http://www.lenholgate.com/zips/JCache8.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache8.zip']);">JCache8.zip</a> we implement a NullCache. The object conforms to the Cache interface standard but simply doesn't bother to cache any of the objects that are given to it. Now you can change the property file to:</p>


<pre class="brush: text gutter: false">com.jetbyte.cache7.CacheManager.MYCACHE.CacheClass=com.jetbyte.cache7.NullCache</pre>


<p>and caching is turned off! The best thing about this change is that it's possible to allow for the special case of "not caching" without having to have any special logic in the application! Not caching is just another kind of caching.</p>

<p><b>Easy cache configuration</b><br />
Now that our server is using properties that are merged in with the system properties to configure its cache without the need for recompilation we may as well configure the caches themselves via properties. In <a href="http://www.lenholgate.com/zips/JCache9.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache9.zip']);">JCache9.zip</a> we allow the scanEveryMillis and entryTimeoutMillis variables to be configured on a per named cache basis from properties.</p>

<p><b>Download</b><br />
The following source was built using Ant and tested with JacORB 3.21 - the open source Corba Java ORB.</p>

<p><a href="http://www.jacorb.org/">Get JacORB</a></p>

<p><a href="http://www.lenholgate.com/zips/JCache1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache2.zip']);">JCache1.zip</a> - a simple HashMap cache</p>

<p><a href="http://www.lenholgate.com/zips/JCache2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache2.zip']);">JCache2.zip</a> - a better key</p>

<p><a href="http://www.lenholgate.com/zips/JCache3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache3.zip']);">JCache3.zip</a> - use an interface</p>

<p><a href="http://www.lenholgate.com/zips/JCache4.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache4.zip']);">JCache4.zip</a> - purge the least recently used entries</p>

<p><a href="http://www.lenholgate.com/zips/JCache5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache5.zip']);">JCache5.zip</a> - use SoftReferences</p>

<p><a href="http://www.lenholgate.com/zips/JCache6.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache6.zip']);">JCache6.zip</a> - purge entries after a maximum time</p>

<p><a href="http://www.lenholgate.com/zips/JCache7.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache7.zip']);">JCache7.zip</a> - use a cache factory</p>

<p><a href="http://www.lenholgate.com/zips/JCache8.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache8.zip']);">JCache8.zip</a> - turn off caching with a null cache</p>

<p><a href="http://www.lenholgate.com/zips/JCache9.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JCache9.zip']);">JCache9.zip</a> - configuration from properties</p>]]>
    </content>
</entry>

<entry>
    <title>The CORBA Evictor Pattern in Java</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/07/the-corba-evictor-pattern-in-java.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.137</id>

    <published>2001-07-23T00:00:00Z</published>
    <updated>2011-06-23T10:57:24Z</updated>

    <summary>When a CORBA server allows its clients to create and destroy objects one of the recommended ways to handle the object lifetime issues is using the Evictor Pattern. In The Evictor Pattern we solved the problem for C++ servers, here...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Java" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[When a CORBA server allows its clients to create and destroy objects one of the recommended ways to handle the object lifetime issues is using the Evictor Pattern. In <a href="http://www.lenholgate.com/archives/000476.html">The Evictor Pattern</a> we solved the problem for C++ servers, here we do the same for Java servers.]]>
        <![CDATA[<p><b>The problem</b><br />
Due to the way that CORBA deals with object lifetime issues you may find it necessary to have the server control the lifetime of objects created on the server by the client. The kind of objects we're talking about here are the objects that get created for a client when the client calls a method on another object on the server. For example a client may wish to access a collection of objects on a server and to do this may call a method that returns an iterator object. The iterator object has a lifetime that is controlled by the client, the client creates it by requesting it from the server, uses it and eventually destroys it with a call to <code>destroy()</code>. Unfortunately servants on the server can be leaked if the client does not call <code>destroy()</code> and because of this it's best to let the server take control. See the <a href="http://www.google.com/custom?domains=lenholgate.com&amp;q=CORBA+&amp;sitesearch=lenholgate.com&amp;client=pub-6888952347469638&amp;forid=1&amp;ie=ISO-8859-1&amp;oe=ISO-8859-1&amp;safe=active&amp;cof=GALT%3A%23008000%3BGL%3A1%3BDIV%3A%23336699%3BVLC%3A663399%3BAH%3Acenter%3BBGC%3AFFFFFF%3BLBGC%3A336699%3BALC%3A0000FF%3BLC%3A0000FF%3BT%3A000000%3BGFNT%3A0000FF%3BGIMP%3A0000FF%3BFORID%3A1%3B&amp;hl=en">C++ CORBA section</a> for more details of the problems that can occur if you dont let the server manage the lifetime of these objects in some way.</p>

<p>You may be surprised to hear that you have these kind of object lifetime problems with Java. After all, as a Java programmer you're used to objects being garbage collected when they're no longer needed - why dont CORBA servants get cleaned up automatically for you? The problem is that the server needs to hold on to a reference to the servant object until the client doesnt want the object any more. If the client behaves itself then there's no problem, once the client finishes with the object it can tell the server that it no longer requires it and the server can release its last reference and the language run-time can garbage collect the servant object when it wants to. If the client terminates unexpectedly or acts maliciously the the server has no way to manage the references that it is holding on behalf of the clients. One solution to this problem is the evictor pattern.</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>
<b>The Evictor Pattern</b><br />
For an excellent description of the concept behind the evictor pattern see <a href="http://www.amazon.co.uk/exec/obidos/redirect?path=ASIN/0201379279&amp;link_code=as2&amp;camp=1634&amp;tag=ramcom-21&amp;creative=6738">Advanced CORBA Programming with C++ (APC)</a><img src="http://www.assoc-amazon.co.uk/e/ir?t=ramcom-21&amp;l=as2&amp;o=2&amp;a=0201379279" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />. In a nutshell, the server decides how long your object can live, when the server decides your time is up it simply destroys your object for you and your next call fails... It doesn't matter if you forget to release your objects when you're done, they'll get cleaned up eventually. It doesn't matter if you terminate without letting the server know and, well, it doesn't matter if you have to write heaps of client code to handle exceptional cases of object disappearance...</p>

<p><b>You're out...</b><br />
The simplest evictor works along these lines. The server has finite resources, if those resources are about to be exceeded then the server should garbage collect existing objects that haven't been used for a while to make way for new objects. The easiest implementation is probably something that uses a least recently used algorithm for eviction of "inactive" servants and is implemented in terms of a <code>ServantLocator</code>. See the <a href="http://www.google.com/custom?domains=lenholgate.com&amp;q=CORBA+&amp;sitesearch=lenholgate.com&amp;client=pub-6888952347469638&amp;forid=1&amp;ie=ISO-8859-1&amp;oe=ISO-8859-1&amp;safe=active&amp;cof=GALT%3A%23008000%3BGL%3A1%3BDIV%3A%23336699%3BVLC%3A663399%3BAH%3Acenter%3BBGC%3AFFFFFF%3BLBGC%3A336699%3BALC%3A0000FF%3BLC%3A0000FF%3BT%3A000000%3BGFNT%3A0000FF%3BGIMP%3A0000FF%3BFORID%3A1%3B&amp;hl=en">C++ CORBA section</a> for details of the server that this example is based on and to compare the Java implementation with the equivalent C++ implementation.</p>

<p><b>An evicting ServantLocator</b><br />
A servant locator is called by the POA that it's associated with to dispatch method calls to the appropriate servant. By associating a servant locator with a POA we can place code that we've written directly in the call sequence for every method dispatch call to any servant that uses that POA. We are now responsible for mapping the CORBA Object ID to the servant that is providing the implementation for that object. Since we manage the mapping we also manage the lifetime of the servants.</p>

<p>Every time we create a servant that is to be managed by our servant locator we must notify the servant locator of the new object so that the object can be added to the servant locator's mapping table. When the servant locator adds the object to the mapping table it must also generate an Object ID and from the Object ID we can create an object reference that refers to our servant so that we can hand it out to our clients.</p>

<p>When we wish to destroy a servant that is managed by our servant locator the object simply requests that the servant locator remove it from its mapping table. When the servant locator does this there are no more outstanding references to the servant and the run-time can garbage collect the servant.</p>

<p>The evictor pattern comes into play when the servant locator's Object ID to servant mapping table is full. At this time the servant locator must make space in its mapping table for new servants by removing existing servants. Our servant locator uses a timeout algorithm for deciding which servants to evict.</p>

<p>The actual management of the Object ID to servant mapping is performed by the <code>EvictorQueue</code> class. This class maintains a fixed size array for servants and the index into the array is used as part of the Object ID for that servant. The actual array holds instances of the <code>EvictableServant</code> class which contains the servant reference and timestamps that are used for eviction and object identification purposes. Each time that a servant is accessed its last used timestamp is updated and it is moved to the tail of a list that is maintained in least recently used order. When the array of servants is full the least recently used list is scanned for an object that has timedout, all objects that have timed out are evicted from the array and are available for garbage collection by the run-time. If no servants have timedout then an exception is thrown.</p>

<p><b>Comparing the Java and C++ implementations</b><br />
The Java implementation is more intrusive than the C++ implementation due to the fact that the Java servant implementation must derive from the appropriate POA class and cannot also derive from a mix-in base class that provides the common servant locator code. We could use a Tie implementation but that is likely to be as complex.</p>

<p><b>Download</b><br />
The following source was built using Ant and tested with JacORB 3.21 - the open source Corba Java ORB.<br />
<a href="http://www.jacorb.org/">Get JacORB</a><br />
<a href="http://www.lenholgate.com/zips/JEvict1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'JEvict1.zip']);">JEvict1.zip</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>23rd July 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>. </li>
<li>23rd August 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul><p></p>]]>
    </content>
</entry>

<entry>
    <title>CORBA - Keep Alive</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/03/corba---keep-alive.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.136</id>

    <published>2001-03-17T01:00:00Z</published>
    <updated>2010-12-22T13:12:49Z</updated>

    <summary>One way of making a reference counted implementation more robust is to run the keep-alive protocol yourself. We demonstrate this option here....</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        One way of making a reference counted implementation more robust is to run the keep-alive protocol yourself. We demonstrate this option here.
        <![CDATA[<p><b>The problem</b><br />
In the <a href="http://www.lenholgate.com/archives/000476.html">previous article</a> we developed a strategy where the server would destroy objects if the client hadn't used them for a certain period of time. This protects the server from leaked objects and is necessary because CORBA doesn't provide a keep alive mechanism. If a client has a valid reason to hold onto an object and not do anything with it then the only way that they can keep the object active in the server is to poke it every now and then - even if they have no reason to make a method call on the object.</p>

<p><b>Keep yourself alive</b><br />
Since clients may have valid reasons for needing to keep their objects active, but not in use, we may as well give them a method on the object that does nothing at the server end but that can be used to make sure that their objects don't get timed out. </p>
<pre class="brush: cpp gutter: false">interface KeepAlive
{
   void Ping();
};</pre>
<p>The client can now call the <code>Ping()</code> method on the object which need not do anything on the server. The mere fact that the method is called and dispatched via the Servant Locator is enough to ensure that the servant is not evicted. Since the client has no way of knowing how often it needs to call the <code>Ping()</code> method to keep the object alive, we could provide a method to return the object's timeout value, this would allow a client to ensure that they call <code>Ping()</code> often enough to keep their object alive.</p>
<pre class="brush: cpp gutter: false">interface KeepAlive
{
   void Ping();
   long GetTimeout();
};</pre>
<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0">&amp;lt;br /&amp;gt;
</iframe>
This approach means that we're effectively implementing a keep alive protocol on our objects that's driven by the client. Whilst the client keeps using the object, or keeps pinging it to keep it alive, the object stays around on the server. When the client finishes with the object they can destroy it, if they forget, or if they terminate unexpectedly then the object will time out and the server will clean it up. This creates more work for the client programmer, but is necessary as CORBA doesn't provide a keep alive protocol for us.</p>

<p>In <a href="http://www.lenholgate.com/zips/KeepAlive1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'KeepAlive1.zip']);">KeepAlive1.zip</a> we add the interface above to the iterator interfaces and add a client command "ping" to demonstrate them in action. Notice that if you run the server and then run a single client with the ping command the final ping will work correctly - the servant hasn't been evicted. This is intentional and is due to the fact that eviction only occurs when new objects are added to the Servant Locator. If you run one client with the ping command and, in another window, repeatedly run a client with the "it" command you'll see that the ping client is evicted before the final ping can occur. The ping client has timed out, due to lack of use and the servant is evicted from the server.</p>

<p>We could check for eviction on every access to the servant locator, but the eviction overhead would then occur on all method calls. Our current design only incurs the overhead of eviction when an object is added to the Servant Locator.</p>

<p><b>Back to reference counting</b><br />
Now that we have a solution that allows the client programmer control over the lifetime of a server object, but still allows the server programmer to be certain that clients cant leak objects we can use it with the reference counted objects. Updating the reference counting base class is easy enough, <code>TRefCountedBase</code> now inherits from <code>TEvictor&lt;&gt;</code> and simply provides the additional <code>AddRef()</code> and <code>Release()</code> functionality. Unfortunately it doesn't work and exposes a problem in our evictor implementation, first I'll explain the problem and then we'll fix it.</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0130962775&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="250" scrolling="no" marginwidth="0" marginheight="0" frameborder="0">&amp;lt;br /&amp;gt;
</iframe>The original reference counting implementation presented in the first two articles reused the reference count supplied by <code>RefCountServantBase</code> as this saved us storing a second counter with the reference counted object. When we evolved the code from being a Servant Locator for a reference counting implementation to being a Servant Locator for an evictor implementation we kept the use of the <code>RefCountServantBase</code> reference count. This was inappropriate as the evicting Servant Locator held only a single reference to each servant. What's more, if anyone else held a reference to the servant then the evictor wouldn't work as it relied on the object removing itself from the Servant Locator when it was destroyed but it initiated that destruction by removing a reference on the servant object. If the removal of the reference didn't cause the destruction of the servant then the object wasn't evicted ... This problem existed since our first evictor implementation but is made more visible when we expose the reference count to the clients through <code>TRefCountedBase</code>. Now whenever we need to evict a reference counted object due to lack of use we will be evicting a servant which has a reference count of more than 1. The Servant Locator will evict the servant and initiate destruction by removing its reference but as clients still hold references to the object the object doesn't destroy itself.</p>

<p>In <a href="http://www.lenholgate.com/zips/KeepAlive2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'KeepAlive2.zip']);">KeepAlive2.zip</a> we address this problem by removing the reliance on <code>RefCountServantBase</code> for objects that we store in the evictor. Now when we wish to evict a servant we simply call <code>RemoveFromLocator()</code> and the object is removed and deleted by the Servant Locator. <code>TRefCountedBase</code> must now have its own reference count for client side references and when the reference count falls to zero it can ask the Servant Locator to remove and destroy it.</p>

<p><b>Download</b><br />
The following source was built using Visual Studio 6.0 SP3 and tested with <a href="http://www.uk.research.att.com/omniORB/index.html">OmniORB</a> - the open source Corba ORB from AT&amp;T Cambridge. You need to add OMNI_HOME to your environment so that the idl compiler, headers and libraries can be found.</p>

<p>To compile the IDL files you will need to change the path used in the build command for the idl files - well, you will unless you happen to have installed OMNI ORB into exactly the same location as I have... Select the IDL files in the project workspace, right click on them, select settings and edit the path to the OMNIIDL compiler.</p>

<p><a href="http://www.uk.research.att.com/omniORB/index.html">Get OmniORB</a><br />
<a href="http://www.lenholgate.com/zips/KeepAlive1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'KeepAlive1.zip']);">KeepAlive1.zip</a><br />
<a href="http://www.lenholgate.com/zips/KeepAlive2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'KeepAlive2.zip']);">KeepAlive2.zip</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>17th March 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>. </li>
<li>23rd August 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul>]]>
    </content>
</entry>

<entry>
    <title>CORBA - The Evictor Pattern</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/03/corba---the-evictor-pattern.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.135</id>

    <published>2001-03-15T01:00:00Z</published>
    <updated>2010-12-23T09:56:30Z</updated>

    <summary>Since CORBA doesn&apos;t really support reliable reference counting implementations we&apos;ll compare one of the recommended methods of servant life-time management with our reference counted iteration interface....</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        Since CORBA doesn&apos;t really support reliable reference counting implementations we&apos;ll compare one of the recommended methods of servant life-time management with our reference counted iteration interface.
        <![CDATA[<p><b>The problem</b><br />
If you can't trust the runtime to make sure that your clients behave themselves, then the server has to take object lifetime issues into its own hands. The evictor pattern does just that. The server decides how long your object can live, when the server decides you've had enough it simply destroys your object for you and your next call fails... It doesn't matter if you forget to release your objects when you're done, they'll get cleaned up eventually. It doesn't matter if you terminate without letting the server know and, well, it doesn't matter if you have to write heaps of code to handle exceptional cases of object disappearance...</p>

<p>To me, this sort of idea seems to be the kind of thing that someone who "write's the server" would come up with. If you're responsible for the server running forever and never being brought down by malicious clients then you'd develop this kind of siege mentality. It doesn't matter if your server becomes unusable because nobody gets a chance to use the objects you give out for long enough to do any work, at least you wont get overrun by clients who just request objects because they know they can crash your precious server eventually...</p>

<p>Of course it depends how you implement the evictor pattern, but on its own, it's always going to make life easier for the server writer and harder for the person who has to write the clients. The usage patterns of the client determine how impossible it becomes...</p>

<p><b>You're out...</b><br />
The  simplest evictor works along these lines. The server has finite resources, if those resources are about to be exceeded then the server should garbage collect existing objects that haven't been used for a while to make way for new objects. The easiest implementation is probably something that uses a least recently used algorithm for eviction of "inactive" servants and is implemented in terms of a <code>ServantLocator</code>. As we saw in a previous article the <code>ServantLocator</code> gets called before and after each method call is dispatched to a servant object. It's in the ideal place to manage a least recently used list of servant objects. In <a href="http://www.lenholgate.com/zips/Evict1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict1.zip']);">Evict1.zip</a> we use the servant locator that we developed for the reference counting examples as a starting point for an evictor implementation that uses a least recently used list to determine when to evict servant objects.</p>

<p><b>Least recently used...</b><br />
In our reference counting servant locator we have a vector of servant pointers and a list of free spaces within that vector. When a new servant is added to the servant locator we look in the free list and if we find a spare slot in the vector we store a pointer to the servant in that slot. If we don't find a free slot we simply expand the vector to make space for the new servant.</p>

<p align="center"><img alt="Evictor Pattern, least recently used list 1" src="http://www.lenholgate.com/blog/images/Evict1.gif" border="0" /></p>

<p>The evictor implementation works a little differently. For a start, there's a limit on how much we want to expand the array of servants. We also need to know which servant was used last so that we can evict it if we need to free up space for a new servant. To enable us to do this we need to add a list of servant pointers that is adjusted each time a method call occurs on a servant. As each call occurs we need to move the servant the call to the end of the list. This leaves the least recently used servant at the head of the list.</p>

<p align="center"><img alt="Evictor Pattern, least recently used list 2" src="http://www.lenholgate.com/blog/images/Evict2.gif" border="0" /></p>

<p>When we wish to evict a servant object so that we can create room for a new servant we simply take the object that is pointed to by the head of the least recently used list and evict that.</p>

<p>All of the logic for accessing the servant objects from the servant locator, and for evicting servants when we run out of resources, is contained within the <code>LRUList</code> class. The example in <a href="http://www.lenholgate.com/zips/Evict1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict1.zip']);">Evict1.zip</a> allows the server to provide up to 10 of each type of iterator at a time. If more are requested then the least recently used is deactivated. You can experiment with this server using the new client commands,  "itx" - which creates an iterator and deliberately leaks it, i.e. it doesn't call destroy when it's done. If you call the server using the client's itx method for more than 10 times you will begin to see the leaked iterators evicted and their resources released. The server displays the eviction queue after each manipulation to it.</p>

<p><b>Too simple by half</b><br />
Unfortunately the implementation shown in <a href="http://www.lenholgate.com/zips/Evict1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict1.zip']);">Evict1.zip</a> is too simplistic. Now that we allow the server to clean up servants whenever it desires, we need to protect ourselves from a client that attempts to use an object after the server has evicted it. Up until now our object id was simply the index into the servant pointer vector. That doesn't work any more. If the server evicts servant 7 the client wont find out about it until it tries to make a call on the object, when it does so it will use the object id that the server gave it, 7, but inside the server that object id now refers to another servant...</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0130962775&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="250" scrolling="no" marginwidth="0" marginheight="0" frameborder="0">
</iframe>We can get around this problem by constructing a slightly more complex object id. Instead of just including the index into the servant vector, we can include a timestamp as well. The timestamp is the time that the servant was added to the locator and is stored in the servant array along with the servant pointer. Now, when we evict a servant from a slot we reuse the slot index but the timestamp changes. When a client attempts to use an object id for an evicted servant the index portion will likely match an active servant, but the timestamp cannot be the same, this allows us to correctly fail the call by throwing a <code>CORBA::OBJECT_NOT_EXIST</code> exception. The fixed up code is presented in <a href="http://www.lenholgate.com/zips/Evict2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict2.zip']);">Evict2.zip</a> along with client code that is adjusted to make exercising it easier - use the "holdit" client command to obtain and iterator and hold onto it for a number of seconds before using it. You can then obtain and leak, iterators so that the original, held, iterator is evicted. When the client times-out the invalid object id is used and the server throws the exception.</p>

<p>Note that in these examples we use <code>time()</code> to generate the creation timestamp. This only has a resolution of seconds so if the turn over of objects in the server is very rapid we could have problems. We might be better off, under Win32, using <code>GetTickCount()</code> instead.</p>

<p><b>Alternative eviction strategies</b><br />
Evicting due to a fixed number of servants being exceeded is probably too harsh a strategy to use in a production system. No matter what number you choose the system will always get into a state where it starts to thrash (evicting servants before they can be used for useful work) if the number of active clients grows to one more than the number of servants that the server is willing to support. Far better to evict servants that have not been used for a specified period of time. In periods of heavy use the server will support as many servants as required but if objects are left idle - perhaps due to client failure - they will be evicted.</p>

<p>This requires that we associate a second timestamp with the servant. This timestamp is updated each time a method is called on the servant. When deciding whether to evict, we can now examine this timestamp to see if the least recently used servant has been inactive for the appropriate amount of time, if it has, we evict it. The value chosen for the timeout will effect how the eviction strategy works, too short and the client may not be able to do useful work, too long and the server will use more resources.</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>In <a href="http://www.lenholgate.com/zips/Evict3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict3.zip']);">Evict3.zip</a> we change the eviction strategy to use the timeout scheme described above. We now have a maximum number of servants that we wish to support, in this case 10, and a timeout of 5 seconds - both values chosen to simplify the testing and demonstration of the evictor at work. Since we now evict due to lack of use, rather than lack of space we could remove the maximum servant restriction. In this example we leave it in place and refuse new requests for objects if we have reached our limit. The problem with doing this is that the server is then open to denial of service attacks from malicious clients that obtain objects simply to use up server resources. The malicious client can continue to obtain objects until it is refused and then continue to use the objects in some way to prevent them timing out.... The thing is, in the presence of such clients there's not a lot you can do. If you don't limit the number of servants then they can simply continue creating servants until your server fails due to lack of resources. If you evict servants that haven't yet timed out then you're back to the thrashing situation that we had with the previous example and genuine clients will be unable to get any work done anyway.</p>

<p>In the next article we give the client a little more control over exactly when their objects are evicted.</p>

<p><b>Java version</b><br />
For a version of the evictor pattern in Java, see <a href="http://www.jetbyte.com/portfolio-showarticle.asp?articleId=34&amp;catId=1&amp;subcatId=9">here</a></p>

<p><b>Download</b><br />
The following source was built using Visual Studio 6.0 SP3 and tested with <a href="http://www.uk.research.att.com/omniORB/index.html">OmniORB</a> - the open source Corba ORB from AT&amp;T Cambridge. You need to add OMNI_HOME to your environment so that the idl compiler, headers and libraries can be found.</p>

<p>To compile the IDL files you will need to change the path used in the build command for the idl files - well, you will unless you happen to have installed OMNI ORB into exactly the same location as I have... Select the IDL files in the project workspace, right click on them, select settings and edit the path to the OMNIIDL compiler.</p>

<p><a href="http://www.uk.research.att.com/omniORB/index.html">Get OmniORB</a><br />
<a href="http://www.lenholgate.com/zips/Evict1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict1.zip']);">Evict1.zip</a><br />
<a href="http://www.lenholgate.com/zips/Evict2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict2.zip']);">Evict2.zip</a><br />
<a href="http://www.lenholgate.com/zips/Evict3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Evict3.zip']);">Evict3.zip</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>15th March 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>.</li>
<li>17th March 2001 - New, simpler, least recently used list implementation.</li>
<li>18th March 2001 - Fixed a bug in Remove().</li>
<li>23rd August 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul><p></p>]]>
    </content>
</entry>

<entry>
    <title>CORBA - Iteration</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/02/corba---iteration.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.134</id>

    <published>2001-02-09T02:00:00Z</published>
    <updated>2012-03-13T09:06:55Z</updated>

    <summary>A CORBA style method of enumeration can be seen in the iteration interfaces on the CORBA Naming Service. Given the code we&apos;ve already written for the enumeration interface we can easily implement an iteration interface as well as (or, more...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[A CORBA style method of enumeration can be seen in the iteration interfaces on the CORBA Naming Service. Given the code we've already written for the <a href="http://www.lenholgate.com/archives/000469.html">enumeration interface</a> we can easily implement an iteration interface as well as (or, more likely, instead of).]]>
        <![CDATA[<p><b>Setting the scene</b><br />
As we pointed out in <a href="http://www.lenholgate.com/archives/000468.html">a previous article</a>, reference counting without a keep-alive protocol is not especially robust and objects may be leaked on the server. We are assuming, for the time being, that this is acceptable to us and, in this articles and the previous, examine how COM and CORBA handle the problem of allowing the client to control how a sequence of objects is returned to it. </p>

<p>Once we've the CORBA equivalent to the COM enumeration interface we'll make them both more robust by using the <a href="http://www.lenholgate.com/archives/000476.html">Evictor Pattern</a> on the server to ensure that our server never becomes clogged with objects that are no longer used.</p>

<p><b>The problem</b><br />
<iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>As was shown in <a href="http://www.lenholgate.com/archives/000469.html">the previous article</a>, it's often better to allow your clients to determine how many items they process in one go when dealing with collections of objects returned from a server. Last time we showed the COM way of dealing with this problem, albeit in a CORBA environment. This article shows the CORBA way of doing the same thing. The convenient thing is that it's only really the interface that changes, the requirements for the underlying implementation stay pretty much the same...</p>

<p><b>The CORBA solution</b><br />
If you look at the CORBA NamingContext interface you'll find that it has a list() method this is defined something like this:</p>
<pre class="brush: cpp gutter: false">typedef sequence&lt;binding&gt; BindingList;
  
interface BindingIterator;
  
interface NamingContext
{
// stuff that we've omitted...
  
   void list(
      in unsigned long how_many,
      out BindingList bl,
      out BindingIterator it);
};
interface BindingIterator 
{
   boolean next_one(out binding b);
  
   boolean next_n(
      in unsigned long how_many,
      out BindingList bl);
       
   void destroy();
};</pre>
<p>When you wish to iterate through a list of bindings you call <code>list()</code> which allows you to specify how many bindings you want returned in the initial <code>BindingList</code> sequence, any subsequent bindings can then be accessed though the <code>BindingIterator</code>. If you ask for more than the available number of bindings then you will get all of them in your initial sequence and the <code>BindingIterator</code> will be <code>nil</code>.</p>

<p>Implementing such an interface for our server is relatively easy now that we have all of the infrastructure in place to support the COM style enumeration interface.</p>

<p>In <a href="http://www.lenholgate.com/zips/Enum5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum5.zip']);">Enum5.zip</a> we add a CORBA style iteration interface to the server. The client code for the iteration interface is more complex than the COM style enumeration due to the fact that the initial sequence needs to be processed and then subsequent sequences are obtained via the iteration interface. Notice that since we only need a <code>delete()</code> method and not fully featured reference counting we can revert to one of the methods used half-way through the reference counting articles...</p>

<p>At first the flexibility in the interface looks useful. The client can ask for all of the sequence in one go and avoid one additional call to the server. However, unless the client knows how many items there are in the sequence, and unless it is sure that it will always be able to handle that number of items, it must also have code to handle the overflow situation. It's actually easier to always pass 0 for "how_many" in the initial call to <code>list()</code> as this means that you only need to write code to handle the iteration interface.</p>

<p><b>Read-Only Iteration</b><br />
In <a href="http://www.lenholgate.com/zips/Enum6.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum6.zip']);">Enum6.zip</a> we add the corresponding read-only iteration interface. The client code is very similar and the server code has already been written for the enumeration interface.</p>

<p><b>Comparing COM and CORBA</b><br />
So. Given the two methods of allowing the client some control over how to retrieve a sequence containing an unknown number of objects, which is best... </p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0130962775&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="250" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>The COM version is flexible but potentially requires at least one more trip to the server than the CORBA method. The CORBA way is more flexible for the client, at the expense of more code needing to be written to take advantage of the added flexibility. A simple destroy method is fine if you can always be sure who owns the interface, whilst fully fledged reference counting means you can pass interfaces about with abandon - if you take care to strictly abide by the rules - and if your reference counting is more robust than the one shown here. Being able to clone an iterator may be useful in some circumstances, but could be implemented in both styles of interface. At the end of the day, they're not that different, they both work and they both require similar support on the server side...</p>

<p>At present, both of these interfaces aren't as robust as they could be. Due to the lack of a keep-alive protocol in CORBA the server can be left with objects that will never be destroyed if their clients terminate without releasing all of their references. We'll address this problem in the next article when we look at the <a href="http://www.lenholgate.com/archives/000476.html">Evictor Pattern</a> and how, in some circumstances, you can give the server control over the lifetime of a client created object.</p>

<p><b>Download</b><br />
The following source was built using Visual Studio 6.0 SP3 and tested with <a href="http://www.uk.research.att.com/omniORB/index.html">OmniORB</a> - the open source Corba ORB from AT&amp;T Cambridge. You need to add OMNI_HOME to your environment so that the idl compiler, headers and libraries can be found.</p>

<p>To compile the IDL files you will need to change the path used in the build command for the idl files - well, you will unless you happen to have installed OMNI ORB into exactly the same location as I have... Select the IDL files in the project workspace, right click on them, select settings and edit the path to the OMNIIDL compiler.</p>

<p><a href="http://www.uk.research.att.com/omniORB/index.html">Get OmniORB</a><br />
<a href="http://www.lenholgate.com/zips/Enum5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum5.zip']);">Enum5.zip - an iteration style interface</a><br />
<a href="http://www.lenholgate.com/zips/Enum6.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum6.zip']);">Enum6.zip - read only iteration</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>9th February 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>. </li>
<li>23rd August 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul>]]>
    </content>
</entry>

<entry>
    <title>CORBA - Enumeration</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/02/corba---enumeration.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.133</id>

    <published>2001-02-09T01:00:00Z</published>
    <updated>2012-03-13T09:05:56Z</updated>

    <summary>CORBA provides sequences as a way of returning collections of items from an method call. The problem with just using unbounded sequences is that the client has no control over how many items it receives as a result of the call. COM gets around this problem using the IEnum style interfaces that allow a client to control how it accesses the items in a collection.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[CORBA provides sequences as a way of returning collections of items from an method call. The problem with just using unbounded sequences is that the client has no control over how many items it receives as a result of the call. COM gets around this problem using the <code>IEnum</code> style interfaces that allow a client to control how it accesses the items in a collection.]]>
        <![CDATA[<p><b>Setting the scene</b><br />
As we pointed out in <a href="http://www.lenholgate.com/archives/000468.html">the last article</a>, reference counting without a keep-alive protocol is not especially robust and objects may be leaked on the server. We'll assume for the time being that this is acceptable to us and, in this articles and the next, examine how COM and CORBA handle the problem of allowing the client to control how a sequence of objects is returned to it.</p>

<p>Once we've presented our enumeration and iteration interfaces we'll make them more robust by using the <a href="http://c2.com/cgi/wiki?EvictorPattern">Evictor Pattern</a> on the server to ensure that our server never becomes clogged with objects that are no longer used.</p>

<p><b>The problem</b><br />
CORBA provides sequences as a way of returning collections of items from an method call. The problem with just using unbounded sequences is that the client has no control over how many items it receives as a result of the call. COM gets around this problem using the <code>IEnum</code> style interfaces that allow a client to control how it accesses the items in a collection, in the following article we'll implement a COM style <code>IEnum</code> interface in CORBA and compare the client memory requirements against simply returning a sequence.</p>

<p><b>Sequences considered harmful</b><br />
<iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0130962775&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="250" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>
In the articles about reference counted CORBA objects we developed a server that could return a sequence of named counter objects. The problem with this style of interface is that the client has no control over how many objects will be returned. The interface works fine when we have few counter objects to return but can cause problems if there are many of them and the client is operating in a situation where the memory available to it is restricted.</p>

<p>In <a href="http://www.lenholgate.com/zips/Enum1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum1.zip']);">Enum1.zip</a> we take the server developed in the reference counting articles and add some code so that you can specify the number of counters that are automatically generated. This allows us to test the client with a server that contains thousands of counters. The client code has been changed so that it does some Win32 specific memory usage calculations so that we can show how much memory is used by a call to the server's <code>GetAll()</code> method that returns a sequence of all of the counters that the server contains. We'll use this example as the basis for adding an enumeration interface and compare the memory usage and client side code required for each approach.</p>

<p>If you run the server without any command line arguments it will generate 1000 named counters by default. You can also specify the number of counters on the command line, but I found that 1000 was about the right number to show the differences in client memory consumption. Once the server is running, run the client with the list command line parameter and you'll see the list of named counters displayed. Once the list has finished displaying you'll get some information on the amount of memory used. This display compares the number and size of the blocks in the heap before and after the call to the server. By experimenting with different numbers of counters in the server you will see that as the number of counters is increased so the memory used in the client goes up. The important thing to realise is that the client has no way of controlling how many counters are returned in a call to <code>GetAll()</code> and because of this there is no way to know how much memory may be required. This could lead to problems if the client must process all of the items on the server but doesn't have enough memory available to do so. Of course, if the server interface were more flexible then the client could process the items in batches of a size that it knows it can handle...</p>

<p><b>IEnumXXXX</b><br />
A COM programmer would get around the problem above by using an enumeration interface. In COM there is a standard manner for returning a collection of objects and that is to use an interface that resembles the <code>IEnumXXXX</code> style of interface shown below:</p>
<pre class="brush: cpp gutter: false">interface IEnumXXXX : IUnknown
{
   HRESULT Next(
      [in] ULONG celt,
      [out] IXXXX **rgelt,
      [out] ULONG *pceltFetched);
  
   HRESULT Skip(
      [in] ULONG celt);
  
   HRESULT Reset();
  
   HRESULT Clone(
      [out] IEnumXXXX **ppenum);
};</pre>
<p>Rather than returning a sequence, or the COM equivalent, a method call will return an interface similar to the one shown above. This allows the client to decide how many objects to retrieve on each call to <code>Next()</code>. The client is fully in control of how much memory can be used whilst they are processing the sequence of objects.</p>

<p>Note that the additional methods allow the client to skip a number of elements, reset the enumeration to the start and to create a copy of the enumerator which is positioned at the same point in the sequence as the original interface. Of course, being a COM interface it's derived from <code>IUnknown</code> and has the standard methods for reference counting and interface discovery.</p>

<p>A CORBA version of the above interface, specialised for the named counter objects used in our server could look something like this.</p>
<pre class="brush: cpp gutter: false">typedef sequence&lt;namedcounter&gt; counterSeq;
  
interface EnumNamedCounter
{
   void AddRef();
  
   void Release();
  
   long Next(
      in unsigned long maxCounters, 
      inout counterSeq theCounters);
     
   boolean Skip(
      in unsigned long numToSkip);
  
   void Reset();
  
   EnumNamedCounter Clone();
};</pre>
<p>Note that we've made the reference counting methods explicit as we don't inherit from any other interfaces. The reference counting is required by an enumeration interface as the server creates the enumeration object especially for a particular client - the enumeration object effectively contains a cursor into the sequence that's being traversed and so is caller specific. What's more, since the client itself can create more copies of these objects by calling <code>Clone()</code> it's important that the client can let the object know when it's no longer required and can clean itself up. Of course, we could ignore reference counting here and just have a "destroy" method, but we'll leave that for later...</p>

<p>Implementation of the above interface is shown in <a href="http://www.lenholgate.com/zips/Enum2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum2.zip']);">Enum2.zip</a> and is relatively straight forward. We use the reference counting template base class that we developed in <a href="http://www.lenholgate.com/zips/RefCounted8.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted8.zip']);">RefCounted8.zip</a> which means we can focus on the implementation of the enumerator. We also take the approach of using the simplest approach that could possibly work. Each enumerator object is initialised by passing in the server's container of counters the enumerator simply iterates through the server's counter container and adds each counter to its own internal list. This is very much a brute force approach but it allows us to add an enumeration interface to the server with very little in the way of code changes and it's good enough for us to be able to use from the client to explore the differences in memory usage on the client side. Of course, the memory usage on the server side is less than ideal as a list of pointers to counters is created inside every enumeration object that the server hands out. We'll address that issue in a later build of the server.</p>

<p>If you run a client with the new server you can compare the memory usage on the client between doing a list, which returns the sequence of objects in one go and an enum where you can specify the number of objects to be returned with each call to <code>Next()</code> on the <code>IEnum</code> interface.</p>

<p>
<iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>
The client code is slightly more complex but it's worth it for having a client that can work with any number of items in the sequence rather than having one that may crash if memory becomes tight. It's also possible to make the client code cleaner by using a technique that I discussed in an <a href="http://www.jetbyte.com/portfolio-showarticle.asp?articleId=23&amp;catId=1&amp;subcatId=3">earlier article</a> on COM enumeration that makes it possible to use STL style iterators with an enumeration interface.</p>

<p><b>A better implementation</b><br />
Our first implementation of the server side enumeration object is pretty crude. We copy the server's internal list and provide our own cursor into it. It might seem that we should just be able to reference the server's counter collection and use an iterator onto it as our cursor. The advantage of this approach is that the sequence that we're enumerating is fixed at the time that the enumerator is created. Enumerating the entire sequence, calling reset and enumerating the sequence again will give exactly the same results even if the server has had counters added or removed whilst we were enumerating.</p>

<p>For some situations there's no other way to implement an enumerator than this brute force method. If we're prepared to relax our requirements somewhat so that we accept that the enumeration will represent he current state of the collection of counters on the server at the time that the <code>Next()</code> method is called then we could, potentially access the server's collection directly. Unfortunately we need to change the server's collection to do this as at present we're using a <code>std::map</code> and iterators into that structure are invalidated by insertions and deletions.</p>

<p>What we need is a container on the server that presents the <code>std::map</code> interface that the server currently uses for looking up individual counters, and also presents an interface that's suitable for iterating over the contents of the map even if the map is added to or deleted from during the iteration.</p>

<p>In <a href="http://www.lenholgate.com/zips/Enum3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum3.zip']);">Enum3.zip</a> we use a templatised counter container that presents all of the required interfaces and fulfils our requirements for not invalidating our iterator whilst we're traversing the collection. This collection contains a <code>std::map</code> and exposes parts of the map's interfaces so that it can be plugged into the server with few code changes. Internally however it's quite different and the map stores the name of the counter and a pointer to a node in the collections internal list of counters rather than a pointer to the counter itself. This allows us to remove a counter from the server's map of counters but still have that counter available for iteration if required. It happens that the only time this situation occurs is when an enumeration interface's cursor is currently at the item that is being deleted.</p>

<p>The advantages of this code is that the enumeration object doesn't need to copy the server's collection of objects when it's created, this leads to faster object creation and less memory used on the server. The collection represented by the enumeration is dynamic, it reflects the state of the collection within the server but because of how the list is implemented in the server the enumeration objects can safely hold onto pointers to nodes in the list and step through the list even if the next node has been deleted by another client...</p>

<p><b>Know your clients</b><br />
There are a lot of round-trips occurring in this simple example. The client is using the object's in a read-only fashion, simply displaying their data, but the server is providing fully fledged object references for the client to work with. This means that for each object the client must make a server call to discover the name of the counter and another to fetch the value and another to release the object once it has finished with it. In this situation it would be advantageous for the server to provide a read-only enumeration or list interface that simply returns the data that's stored in the objects concerned rather than the object references.</p>

<p>By defining a struct that contains the name and value of a named counter we can declare a sequence of these structs and then return this sequence of data rather than the sequence of object references. We add this functionality to the client and server in <a href="http://www.lenholgate.com/zips/Enum4.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum4.zip']);">Enum4.zip</a></p>

<p>If you now compare the performance of the "list" and "listro" commands on the client you will see that the "listro" command returns data much faster than the "list" command. This is because all of the data copying is done on the server and then it's marshaled across the client in one call. You'll also notice that there's actually less memory used on the client when we do things this way. If we then compare the "enum" and "enumro" calls we'll see the same speed increases and memory reductions.</p>

<p>Of course, the exact amount of memory used on the client will vary with the data that's being retrieved but this just goes to show that in certain circumstances it is more efficient to allow a client access to all of the data they want in one hit, rather than requiring them to use an object reference to retrieve the data one piece at a time.</p>

<p><b>The CORBA way</b><br />
The examples above show how to implement a COM style enumeration interface. There is a slightly more "standard" CORBA way to achieve a similar thing. The Naming Service returns sequences using CORBA iteration interfaces, we'll compare these to the COM style enumeration interfaces in the next article.</p>

<p><b>Download</b><br />
The following source was built using Visual Studio 6.0 SP3 and tested with <a href="http://www.uk.research.att.com/omniORB/index.html">OmniORB</a> - the open source Corba ORB from AT&amp;T Cambridge. You need to add OMNI_HOME to your environment so that the idl compiler, headers and libraries can be found.</p>

<p>To compile the IDL files you will need to change the path used in the build command for the idl files - well, you will unless you happen to have installed OMNI ORB into exactly the same location as I have... Select the IDL files in the project workspace, right click on them, select settings and edit the path to the OMNIIDL compiler.</p>

<p><a href="http://www.uk.research.att.com/omniORB/index.html">Get OmniORB</a><br />
<a href="http://www.lenholgate.com/zips/Enum1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum1.zip']);">Enum1.zip - shows the memory issues in returning a sequence</a><br />
<a href="http://www.lenholgate.com/zips/Enum2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum2.zip']);">Enum2.zip - a simple enumeration implementation</a><br />
<a href="http://www.lenholgate.com/zips/Enum3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum3.zip']);">Enum3.zip - don't copy objects on the server</a><br />
<a href="http://www.lenholgate.com/zips/Enum4.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'Enum4.zip']);">Enum4.zip - read only enumeration of data rather than objects</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>9th February 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>. </li>
<li>17th August 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul>]]>
    </content>
</entry>

<entry>
    <title>CORBA - Reference Counting Issues</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/02/corba---reference-counting-issues.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.132</id>

    <published>2001-02-09T00:00:00Z</published>
    <updated>2013-05-01T08:01:49Z</updated>

    <summary>At the end of the second article we have developed a self contained reference counting implementation that appears to work. Unfortunately, it&apos;s still far from reliable as CORBA doesn&apos;t provide the level of support for reference counting that&apos;s built into COM. In this article we discuss the problem and the various CORBA methods for controlling server object lifetime.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[At the end of the <a href="http://www.lenholgate.com/archives/000467.html">second article</a> we have developed a self contained reference counting implementation that appears to work. Unfortunately, it's still far from reliable as CORBA doesn't provide the level of support for reference counting that's built into COM. In this article we discuss the problem and the various CORBA methods for controlling server object lifetime.]]>
        <![CDATA[<p><b>The problem</b><br />
In the last <a href="http://www.lenholgate.com/archives/000343.html">two</a> <a href="http://www.lenholgate.com/archives/000467.html">articles</a> we've worked our way towards implementing a COM style reference counting system in CORBA. The final example works pretty well, is easy to add to an existing servant object and appears to solve our problem. The main problem is that although it looks like we're doing everything we would be doing under COM we don't have the same support from the COM run-time in CORBA and because of this any form of effective and safe reference counting is very difficult to achieve within a CORBA system.</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0130962775&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="250" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>Firstly the COM run-time manages the creation of the proxy and stub code used when using an out of process object, this means that COM has an intimate knowledge of who is holding references to which objects. Secondly, when using COM between machines the runtime sends keep-alive messages, or pings, between machines. This allows the COM run-time on each machine to know when another machine is no longer reachable and release any references held by clients on that machine. CORBA doesn't do either of these things and it's very hard in a CORBA system to know when a client has gone away. What's more, CORBA object references can be saved as a string and then passed between clients without the server knowing what's going on - this is similar to being able to save a COM interface pointer in a stream and un-stream it any where, any number of times.</p>

<p>As you can see, although our latest example code works if the client is well behaved and reliable it can't be guaranteed to work if the client deliberately tries to subvert the reference count or if the client or client's network connection terminate unexpectedly.</p>

<p><b>The DCOM ping</b><br />
CORBA doesn't impose a life-time management protocol on you. COM does. When using COM between different machines the DCOM run-time sends keep-alive messages at periodic intervals on a per machine basis. That means that only one ping message is sent per machine connected to a server and the same packet is used  for all client connections between those machines. It doesn't matter if a client has 100 connections to server objects, only one packet needs to be sent to keep all of the client connections alive. What's more, DCOM uses delta pinging. This means that to minimise the size of the ping packet sent between machines, only the differences between the last successful ping message are sent. Finally COM piggybacks the ping message onto normal messages between the client and server so it's only if a client is completely inactive that it actually sends periodic ping messages (at 2 minute intervals).</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>
So, the COM lifetime management overhead is fairly low and provides a useful service. Unfortunately this is not an option with CORBA. CORBA doesn't do any lifetime management for you. If you want it you have to do it yourself. The people who don't need lifetime management and who want to scale their systems to a great size think this is cool. They don't want CORBA doing anything behind their backs. They don't want one extra message per client machine every 2 minutes. Whilst I agree that it would be good if you could configure the DCOM ping so that you could easily turn it off, on a per object basis, if you didn't need it, I'm surprised that CORBA doesn't provide any support for this kind of thing. With the highly configurable object activation system that's provided by the POA, it would seem that it would be easy to have some form of client/server keep-alive system as an option.</p>

<p><b>CORBA solutions</b><br />
So, robust reference counting solutions aren't as easy as they look on CORBA due to the lack of support you get from the underlying framework. Basically, you have to do any keep-alive stuff that you might want yourself, in user code.</p>

<p>There are several ways to do this but the favorite amongst CORBA types seems to be using the Evictor Pattern on the server. Rather than allowing the client to control the lifetime of a server object as you would if you used reference counting, you allow let the server control it. The idea is that the server can decide that your object is no longer needed and dispose of it for you. How inconvenient that is to you as a client depends on how the person who designs the server picks the algorithm that they use to decide when to dispose of objects. In the absence of any keep-alive protocol, this method is, at best, the lesser of two evils. Assuming a sensible algorithm is chosen the server should never get overrun with objects and die due to lack of resources. Of course, it may well become unusable if it gets to a point where it begins to thrash and dispose of objects before clients can do useful work with them. The problem is that the Evictor Pattern must always result in more complex code on the client. Even if the server's eviction algorithm is published as part of the interface specification and doesn't change, the client code could fall foul of it and should probably be able to handle the need to request another object and resynchronize itself with the newly created state of that object.</p>

<p>One problem with the Evictor Pattern is that it doesn't work in all situations where, if you were a COM person, you'd usually use reference counting. For example, we can't replace our use of reference counting in the previous examples with a pure Evictor implementation as the objects life times could be completely random... In the next two articles we'll discuss implementing COM IEnum style enumeration interfaces in CORBA and then their CORBA equivalent, iteration interfaces. We'll assume that the level of safety given by our current reference counting implementation is sufficient, for the time being, and then adapt these objects to use the Evictor Pattern - as they are suited to it.</p>

<p>If we wanted to continue towards a truly robust reference counting implementation then we will have to do the keep-alive protocol ourselves. This can be driven from the client end, by using explicit keep-alive messages or a combination of keep-alive messages and normal method calls with the server evicting the object if it hasn't been touched for a certain period of time; or from the server end, with reverse keep alive where the client hands out a callback interface either once, or with with every reference counted object it requests, and the server then periodically pings its clients - perhaps just before it attempts to evict their object.</p>

<p>The thing to remember is, if the run-time doesn't help you then simple reference counting schemes can't be robust.</p>

<p><b>Revision history</b><br />
</p><ul>
<li>9th February 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>. </li>
<li>17th August 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul>
<p></p>]]>
    </content>
</entry>

<entry>
    <title>CORBA - More Reference Counting</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/02/corba---more-reference-counting.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.131</id>

    <published>2001-02-06T01:00:00Z</published>
    <updated>2010-12-22T13:18:37Z</updated>

    <summary>Although we managed to develop a working solution in the first CORBA reference counting article the results were ugly and fragile. In this article we attempt to clean things up a little and, in doing so, get intimate with the Portable Object Adapter and its Servant Managers.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[Although we managed to develop a working solution in the <a href="http://www.lenholgate.com/archives/000343.html">first CORBA reference counting article</a> the results were ugly and fragile. In this article we attempt to clean things up a little and, in doing so, get intimate with the Portable Object Adapter and its Servant Managers.]]>
        <![CDATA[<p><b>The problem</b><br />
In the <a href="http://www.lenholgate.com/archives/000343.html">previous article</a> we started out with the goal of adding COM style reference counting to a CORBA object. We found that the obvious COM way to do this didn't work due to the way that the default behavior for the POA prevents us from deleting a servant object from within a method call on that servant object. Although the article was written from the point of view of reference counting the servant object, it is equally valid for any object where you want the client of the object to have some say over when the servant is deleted. We worked our way towards a solution, but it was fragile and intrusive. This article attempts to remedy the problems we encountered in the last article and provide a robust and easy to use reference counting base class for servant objects. Once we appear to achieve that goal the next article will expose the flaws that mean that allowing the client control over server object life-time in CORBA is not usually a good idea.</p>

<p><b>My own POA</b><br />
<iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0130962775&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="250" scrolling="no" marginwidth="0" marginheight="0" frameborder="0">
</iframe>
The servant object that we finally ended up with in <a href="http://www.lenholgate.com/zips/RefCounted4.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted4.zip']);">RefCounted4.zip</a> required that you passed in the POA that you wanted the servant to be activated in. This cluttered the servant object's constructor somewhat and made it possible to pass in the wrong kind of POA which would cause the servant to fail. Since we know the type of POA that we require (the POA must have the <code>RETAIN</code> policy set and it shouldn't have a custom servant locator or servant activator assigned to it) we could just as easily have the object itself manage the association with its POA and keep all of the details internal to the object.</p>

<p>In <a href="http://www.lenholgate.com/zips/RefCounted5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted5.zip']);">RefCounted5.zip</a> we add a static data member to the reference counted class. This holds the POA that's to be used by all instances of the servant object. The POA static data member is accessed via a private member function, GetPOA(). The job of this function is to create the correct type of POA the first time it's called, store a reference to that POA in the static data member and from then on hand out that reference when asked. This allows all of the servant objects to share the same POA and removes the need to have the correct type of POA supplied when you create the servant objects.</p>

<p>This change makes the code slightly more robust and less complicated as it's no longer possible to pass in an inappropriately configured POA. The code is still more complex than it needs to be. It still has to jump through hoops to avoid the way that the POA we're using wants to fiddle with our servant objects after we've decided they're no longer needed.</p>

<p><b>Servant Locators</b><br />
CORBA lets you configure how your servant objects are hooked up to your CORBA Objects. You can customise a POA and inject your own code into the object activation and deactivation phases. If we could provide our own implementation of the activation, deactivation and object ID to servant lookup code then we would be free of the problems posed by the default POA behavior as we wouldn't require the POA to maintain an Active Object Map for us.</p>

<p>To do this we need to write a Servant Locator. This is a class derived from <code>POA_PortableServer::ServantLocator</code> that has a method called <code>preinvoke</code> called before every method call dispatched onto the object and a method called <code>postinvoke</code> called after each call completes. The call to <code>previnvoke</code> is supplied with an object ID and it's expected to find, or create, the appropriate servant object for the corresponding CORBA object and return it. The POA then dispatches the method call onto the servant and calls the servant locator's <code>postinvoke</code> method. The <code>postinvoke</code> method is passed the object ID and the servant and it can do what it likes with them.</p>

<p>
<iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>
Our implementation of a servant locator maintains a vector of pointers to servant objects where the object ID of the associated CORBA object corresponds to the position of the servant object in the vector. The locator also maintains a list of free slots in the servant object vector so that it can reuse spaces in the vector as servants are deleted. When a servant object is created it adds itself into the servant locator that's associated with its POA and when it's deleted it removes itself from the associated locator.</p>

<p><b>Configuring the POA</b><br />
To use a servant locator in a POA we need to create the POA with the following policies: A request processing policy of <code>USE_SERVANT_MANAGER</code> - which means that the POA will call our servant manager, in this case a servant locator. A servant retention policy of <code>NON_RETAIN</code> - which prevents the POA from maintaining an Active Object Map on our behalf. And an ID assignment policy of <code>USER_ID</code> - which allows us to manage the generation of object IDs for the CORBA objects that are activated in the POA.</p>

<p>Because of the way that we arranged the code in <a href="http://www.lenholgate.com/zips/RefCounted5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted5.zip']);">RefCounted5.zip</a> we simply need to adjust the <code>GetPOA()</code> call so that it creates an appropriately configured POA and adds our servant locator to it.</p>

<p>This code is presented in <a href="http://www.lenholgate.com/zips/RefCounted6.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted6.zip']);">RefCounted6.zip</a>. Notice how the reference counting methods inside the reference counted object are now much simpler, we're effectively hijacking the code that's used for server side reference counting and using for client side references too.</p>

<p><b>Reusing the code</b><br />
All of the code required for adding reference counting using a servant locator and appropriately configured POA is common to all servant objects that need to be reference counted. In <a href="http://www.lenholgate.com/zips/RefCounted7.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted7.zip']);">RefCounted7.zip</a> we factor all of this common code out into a reusable template base class that handles all of the nitty gritty details for us. All that is required is for our class that requires reference counting to derive from the template base class. In addition, we factor out the reference counting members of the interface into their own interface so that we can simply derive any interfaces that require reference counting from this base interface.</p>

<p>By now, it seems that the one thing missing from our template is a guarantee of thread safety. We fix that with <a href="http://www.lenholgate.com/zips/RefCounted8.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted8.zip']);">RefCounted8.zip</a> by adding a mutex to the base class and locking and unlocking the mutex around each access to the servant locator's shared resources - the vector and the list.</p>

<p>The result is a solution which is neither fragile nor invasive. The servant object needs no changes made to incorporate reference counting except for the inclusion of the correct base class in its inheritance graph.</p>

<p><b>The end?</b><br />
So, we've achieved our aim, or at least it looks that way. Unfortunately although this solution looks pretty good it's likely to result in references being leaked by misbehaved clients and server side objects existing well beyond their use by date... The <a href="http://www.lenholgate.com/archives/000468.html">next article</a> explains the problems that still exist and why, in CORBA, it's very difficult to fix them.</p>

<p><b>Download</b><br />
The following source was built using Visual Studio 6.0 SP3 and tested with <a href="http://www.uk.research.att.com/omniORB/index.html">OmniORB</a> - the open source Corba ORB from AT&amp;T Cambridge. You need to add OMNI_HOME to your environment so that the idl compiler, headers and libraries can be found.</p>

<p>To compile the IDL files you will need to change the path used in the build command for the idl files - well, you will unless you happen to have installed OMNI ORB into exactly the same location as I have... Select the IDL files in the project workspace, right click on them, select settings and edit the path to the OMNIIDL compiler.</p>

<p><a href="http://www.uk.research.att.com/omniORB/index.html">Get OmniORB</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted5.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted5.zip']);">RefCounted5.zip - manage our own POA</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted6.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted6.zip']);">RefCounted6.zip - use a servant locator</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted7.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted7.zip']);">RefCounted7.zip - factor all the code into a reusable template</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted8.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted8.zip']);">RefCounted8.zip - make the servant locator thread safe</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>6th February 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>. </li>
<li>15th March 2001 - adjusted the project files so that they use the OMNI_HOME environment variable to pick up the idl compiler, headers and libraries. </li>
<li>17th August 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul><p></p>]]>
    </content>
</entry>

<entry>
    <title>CORBA - Reference Counting</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2001/02/corba---reference-counting.html" />
    <id>tag:www.socketframework.com,2001:/blog//12.130</id>

    <published>2001-02-06T00:00:00Z</published>
    <updated>2013-05-01T07:59:51Z</updated>

    <summary>We&apos;ve been developing code on Windows and Unix for quite some time, but most of the distributed component work we&apos;ve been involved in has been done using COM. That&apos;s changing now as clients have more requirements for Unix based component solutions. We&apos;re currently evaluating CORBA ORBs and learning how the CORBA way of doing things differs from the COM way. 

The first difference that we came across was the way that CORBA servers deal with object lifetime issues. Adding reference counting to CORBA objects isn&apos;t as easy as it first seems

Please note that these articles initially explore COM concepts implemented in CORBA, some of the early implementations, whilst working examples, aren&apos;t at all production quality... If you want something reliable, that works, skip forward to the Evictor Pattern.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="CORBA" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>We've been developing code on Windows and Unix for quite some time, but most of the distributed component work we've been involved in has been done using COM. That's changing now as clients have more requirements for Unix based component solutions. We're currently evaluating CORBA ORBs and learning how the CORBA way of doing things differs from the COM way. </p>

<p>The first difference that we came across was the way that CORBA servers deal with object lifetime issues. Adding reference counting to CORBA objects isn't as easy as it first seems</p>

<i>Please note that these articles initially explore COM concepts implemented in CORBA, some of the early implementations, whilst working examples, aren't at all production quality... If you want something reliable, that works, skip forward to the Evictor Pattern.</i>]]>
        <![CDATA[<p>In COM you're used to your clients having control over the lifetime of your server objects, reference counting is a way of life and if you need an object that acts as a persistent singleton within a server you can achieve it by altering how the reference counting works in the server. With CORBA the opposite stance is taken. Server object lifetime is completely unrelated to client proxy lifetime. In an ideal, non-deterministic world, some form of garbage collection on the server might reclaim objects for you. Whilst the CORBA books and FAQs admit that you can implement reference counting if you really have to, they never seem to give any examples of how to do it. In fact, adding reference counting to CORBA isn't that hard once you've realised that the obvious ways just don't work. These articles will show you several ways of doing it and then discuss why you probably don't want to.</p>

<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0130962775&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="250" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>
Be aware that in showing how reference counting could be implemented in CORBA we're going to take the obvious route that a COM programmer might take. In doing so we'll encounter lots of dead ends and frustrations, this is deliberate. The intention is to show that the methods that are so obvious to us as COM programmers just don't cut it in CORBA. Even when we finally get to a point where we have what seems to be a working implementation we'll find out that it's less than ideal. Don't be put off, we do end up with a working and reliable solution to the problem, we just have to approach from a slightly different angle..</p>

<p><b>A simple example</b><br />
Suppose we have a server that hosts a single factory object. The factory object can be asked for instances of a named counter object. Each named counter is just a value with a name associated with it. The IDL for the server could be as follows:</p>
<pre class="brush: cpp gutter: false">interface NamedCounter
{
   readonly attribute string name ;
   attribute long value;
};

interface NamedCounterFactory
{
   exception CounterExists 
   {
      string name;
   };

   exception NoCounterExists 
   {
      string name;
   };

   typedef sequence&lt;namedcounter&gt; counterSeq;

   NamedCounter GetNamedCounter(
      in string name) raises (NoCounterExists);

   void AddNamedCounter(
      in string name, 
      in long initialValue) raises (CounterExists);

   void RemoveNamedCounter(
      in string name) raises (NoCounterExists);

   counterSeq GetAll();
};</pre>
<p><iframe align="right" src="http://rcm-uk.amazon.co.uk/e/cm?t=ramcom-21&amp;o=2&amp;p=8&amp;l=as1&amp;asins=0201379279&amp;fc1=000000&amp;=1&amp;lc1=0000ff&amp;bc1=000000&amp;lt1=_blank&amp;IS2=1&amp;f=ifr&amp;bg1=ffffff" width="120" height="280" scrolling="no" marginwidth="0" marginheight="0" frameborder="0"></iframe>
The server registers the NamedCounterFactory object with the naming service and clients can obtain a reference to it by name. The client can then list the current counters using <code>GetAll()</code>. They can add, remove and lookup counters by name. The client also allows you to hold onto a reference to an object for a period of seconds. Run the client without any command line parameters to find out how to use it. The implementation is fairly simple and a fully working example can be found in the downloadable file <a href="http://www.lenholgate.com/zips/RefCounted1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted1.zip']);">RefCounted1.zip</a>.</p>

<p>There are a few problems with this initial implementation. One is that if you run the server and then run a client and add a counter using the command "client add counter"; then run a client in another window and have it hold onto a counter reference for 10 seconds, using "client hold counter 10". If you then switch back to the first client and run it again to delete the counter, using "client del counter" an exception is thrown on the server because a servant that is currently active has been deleted. Also, when the client that is holding a reference to the now deleted counter times out, it attempts to access the counter and another exception is thrown. This second exception is caused by the counter servant no longer existing in the server.</p>

<p>If we remove the line that deletes the named counter servant object when the client deletes the named counter then these problems go away, but the server is now leaking memory. In a Java implementation the runtime may eventually decide to clean up the leaked objects - but don't count on it, the objects are still being referenced within the server, and when we're working in C++ the memory is gone for good anyway and this could eventually lead to the server crashing due to a low memory situation.</p>

<p><b>Breaking the bond</b><br />
There are at least two things wrong with the approach taken in <a href="http://www.lenholgate.com/zips/RefCounted1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted1.zip']);">RefCounted1.zip</a>. Firstly we're deleting a servant to a CORBA object that is still active. Secondly there's no way of knowing if anyone else is using the CORBA object when we call delete on the servant object. We can attempt to address the first issue by deactivating the CORBA object before we delete the associated servant object. We'll address the second issue with explicit, COM style, client side reference counting a little later.</p>

<p>By default the Portable Object Adapter that is managing our servants has the RETAIN policy value set. This means that the POA maintains an Active Object Map for us. A POA's Active Object Map is an association between the CORBA object's ID and the servant that is implementing that object. It's used by the POA during method call dispatch to locate the appropriate servant object for a given object ID. Having a servant delete itself without the POA's permission means that the POA now has an object ID that maps to a servant that doesn't exist anymore. The POA doesn't know that the servant has been deleted and so probably crashes your program if it tries to route any more calls to the object concerned.</p>

<p>To be able to delete the servant object you must first deactivate the associated CORBA object and thus sever the link between the CORBA object's ID and the servant object that you wish to delete. You can do this by calling <code>deactivate_object</code> on the POA that you activated your object in, and passing the object ID to identify the CORBA object that you wish to deactivate. Your servant object needs to keep track of the POA that it was activated in, and the object ID of the CORBA object that it's providing an implementation for.</p>

<p>In <a href="http://www.lenholgate.com/zips/RefCounted2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted2.zip']);">RefCounted2.zip</a> we adjust the implementation to call <code>deactivate_object</code> in the destructor of the servant object. This removes the exception caused by the CORBA object still being active when the servant object is deleted but as we see later, the POA can still crash our server.</p>

<p><b>AddRef and Release</b><br />
To solve the problem of deleting an object when references to it are being held, requires that we allow the servant object to know how many references to it are in existence and only allow itself to be deleted if the count of outstanding references falls to 0. Note that by doing this we're handing over control of our server-side object to the client, they may never release them, or may acquire more objects than they need in an attempt to bring the server down. Anyway, we'll ignore this for now, in <a href="http://www.lenholgate.com/zips/RefCounted3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted3.zip']);">RefCounted3.zip</a> we alter the interface to the named counter object so that includes support for two explicit reference counting methods.</p>

<p>The resulting IDL now looks like this:</p>
<pre class="brush: cpp gutter: false">interface NamedCounter
{
   readonly attribute string name;

   attribute long value;<p></p>

   void AddRef();
   void Release();
};</pre>
<p>Now, when ever a named counter object has a reference passed out <code>AddRef()</code> should be called and whenever that reference is no longer required <code>Release()</code> should be called. Methods on the named counter factory object that hand out references to a named counter should call <code>AddRef()</code> before returning the reference to the client; the client should call <code>AddRef()</code> each time it makes a copy of a named counter reference and <code>Release()</code> each time it is finished with a reference. This approach will be familiar to us as COM programmers.</p>

<p>The implementation of <code>AddRef()</code> and <code>Release()</code> should be pretty simple.</p>
<pre class="brush: cpp gutter: false">void NamedCounterImpl::AddRef()
{
   m_refCount++;
}

void NamedCounterImpl::Release()
{
   if (--m_refCount == 0)
   {
      delete this;
   }
}</pre>
<p>We'll ignore the lack of thread safe access to the reference count for now, it's the least of our worries.</p>

<p>Unfortunately, for CORBA servants, this approach wont work. The POA is managing the dispatch of method calls to our servant object and it's also managing the Active Object Map. Even if we call <code>deactivate_object</code> to deactivate the object and remove it from the Active Object Map before the servant object is destroyed we will still have problems due to how the POA deals with deactivated objects. Even though we deactivate the object before we delete it, the POA will expect the object to be around until after the method call completes.</p>

<p>A servant that uses the POA must be derived from <code>PortableServer::ServantBase</code>. This provides some base functionality that the POA requires to manipulate the servant object whilst it's managing it. Part of the functionality is to include support for a form of reference counting, this reference counting is purely server side. The POA assumes that our servant object lifetime is managed by the reference counting scheme supplied by <code>PortableServer::ServantBase</code> and calls <code>_add_ref()</code> and <code>_remove_ref()</code> methods when the object is manipulated. The POA calls <code>_add_ref()</code> when the servant object is activated and the corresponding call to <code>_remove_ref()</code> is not made until the servant is deactivated and removed from the Active Object Map. This reference counting is used to allow servant objects to be created on the heap in the server, activated and handed out to clients, and then automatically cleaned up when the server deactivates them. Unfortunately, this isn't quite the kind of reference counting that we require to keep a CORBA object active whilst it has clients that refer to it.</p>

<p>Since the POA doesn't require that all objects are allocated on the heap, the reference counting methods in ServantBase are actually no ops. All servant objects implement them, but unless you derive your servant from <code>PortableServer::RefCountServantBase</code> they do nothing. This allows server developers to choose how the lifetime of their servants is managed. Heap based servants should inherit from <code>RefCountServantBase</code>, stack based servants shouldn't.</p>

<p>When <code>deactivate_object</code> is called the POA prevents any future method calls on the servant object, allows all current method calls to complete, and then deactivates the object. First the servant object is remove from the Active Object Map and then a call <code>_remove_ref()</code> is made to to handle reference counted, heap-based servants derived from <code>RefCountServantBase</code>. Because of this, calling <code>deactivate_object</code> and then deleting the servant will cause an access violation when <code>_remove_ref()</code> is called as the object is deactivated after the method call that did the delete returns to the POA.</p>

<p>It's also worth noticing the changes that are required in the client code when we use explicit reference counting. As a COM programmer we can probably ignore it because it's so normal looking, but we need to release the references to the counter objects returned when we call <code>GetAll()</code> and <code>GetNamedCounter()</code> and if we were to pass references about within the client we would need to call <code>AddRef()</code> in appropriate places too...</p>

<p><b>Using RefCountServantBase</b><br />
It would be nice to be able to implement <code>AddRef()</code> and <code>Release()</code> something like this.</p>
<pre class="brush: cpp gutter: false">void NamedCounterImpl::AddRef()
{
   RefCountServantBase::_add_ref();
}

void NamedCounterImpl::Release()
{
   RefCountServantBase::_release_ref();
}</pre>
<p>After all, the code in <code>RefCountServantBase</code> does the right thing. Unfortunately we need to know when to deactivate the object as deactivating the object removes the final reference. To be able to know that we need to be able to peek at the reference count inside of <code>RefCountServantBase</code>, and we can't do that because it's a private data member. The reference count maintained in <code>RefCountServantBase</code> is for server side references, it's not supposed to be hijacked like this for client side reference counting...</p>

<p>We can't do clever things; like call <code>deactivate_object</code> in the object's destructor as the call to <code>deactivate_object</code> must occur to initiate servant destruction, and the method that made the <code>deactivate_object</code> call must return to the POA before the servant object is actually deleted.</p>

<p>This results in us having to keep our own reference count and do something like this:</p>
<pre class="brush: cpp gutter: false">void NamedCounterImpl::AddRef()
{
   m_refCount++;
}

void NamedCounterImpl::Release()
{
   if (--m_refCount == 0)
   {
      RefCountServantBase::_remove_ref();
      m_poa-&gt;deactivate_object(m_objectID);
}</pre>
<p></p>Notice that the extra call to <code>_remove_ref()</code> is required because we increment the reference count when we first activate our object - the common CORBA way to deal with this is to activate the object and then pass ownership of it to the POA by calling <code>_remove_ref()</code> straight away. So we could move the extra call to <code>_remove_ref()</code> up into the object's constructor, where the object activation occurs, but for some reason that looks really weird to me and I prefer it to be in the release call...<p></p>

<p>The resulting solution is presented in <a href="http://www.lenholgate.com/zips/RefCounted4.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted4.zip']);">RefCounted4.zip</a>. We now have two reference counts, one in <code>RefCountServantBase</code> to handle the lifetime of the servant object based on references held inside the server process and one in our own object to handle client references. In addition, our own object needs to keep track of its object ID and the POA that it was created in - this complicates the object's constructor; and also, the POA that the servant was activated in must have the RETAIN policy set and it shouldn't have a custom servant locator or servant activator assigned to it.</p>

<p>All in all, it seems that the solution we have here is fragile and complicated to manage. It does work though, so it's a start... We'll address some of these problems in the <a href="http://www.lenholgate.com/archives/000467.html">next article</a>.</p>

<p><b>Download</b><br />
The following source was built using Visual Studio 6.0 SP3 and tested with <a href="http://www.uk.research.att.com/omniORB/index.html">OmniORB</a> - the open source Corba ORB from AT&amp;T Cambridge. You need to add <code>OMNI_HOME</code> to your environment so that the idl compiler, headers and libraries can be found.</p>

<p><a href="http://www.uk.research.att.com/omniORB/index.html">Get OmniORB</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted1.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted1.zip']);">RefCounted1.zip - the simple example with no reference counting</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted2.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted2.zip']);">RefCounted2.zip - try deactivating the object first</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted3.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted3.zip']);">RefCounted3.zip - add reference counting methods to the interface</a><br />
<a href="http://www.lenholgate.com/zips/RefCounted4.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'RefCounted4.zip']);">RefCounted4.zip - another attempt</a></p>

<p><b>Revision history</b><br />
</p><ul>
<li>6th February 2001 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>. </li>
<li>15th March 2001 - adjusted the project files so that they use the OMNI_HOME environment variable to pick up the idl compiler, headers and libraries. </li>
<li>21st July 2004 - Reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>. </li>
</ul><p></p>]]>
    </content>
</entry>

<entry>
    <title>COM+ Administration</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2000/06/com-administration.html" />
    <id>tag:www.socketframework.com,2000:/blog//12.129</id>

    <published>2000-06-29T00:00:01Z</published>
    <updated>2010-12-22T13:20:53Z</updated>

    <summary>COM+ applications can be complex to configure and there&apos;s no standard way to save that configuration in a form that can be placed under version control. Luckily the COM+ Catalog is accessible by a set of completely scriptable COM objects....</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        COM+ applications can be complex to configure and there&apos;s no standard way to save that configuration in a form that can be placed under version control. Luckily the COM+ Catalog is accessible by a set of completely scriptable COM objects. These scripts take advantage of this to provide some functionality that should really come as standard.
        <![CDATA[<p><b>Scripting the COM+ Catalog</b><br />
COM+ applications can be complex to configure. This complexity increases as you use more COM+ services. By the time you have an application that consists of several configured components, using multiple roles, and method level security it is almost impossible to easily document the configuration of the application. In an environment where all of your source code is routinely stored in a configuration management system and you can automatically rebuild any version of your application from your source tree, having to then manually reconfigure your COM+ application from scribbled notes and memories is simply not an option.</p>

<p>The COM+ Catalog is manipulated by the COM+ Admin COM objects. These allow you to do anything that can be done using the COM+ Explorer programmatically. The interfaces are very general, dealing in COM+ catalog collections which contain COM+ catalog objects, which in turn have name/value properties. These interfaces can be used to access the entire configuration of your COM+ application, store it externally to the COM+ catalog and then rebuild your application from it.</p>

<p>The scripts available from these pages make it possible to save your COM+ application's configuration as an XML document, place it under version control with the rest of your source code and then recreate your COM+ application from it at a later date.</p>

<p>You can save multiple applications into a single XML document, the result of saving the three COM+ applications that form part of IIS can be seen <a href="http://www.lenholgate.com/zips/iis.xml" onclick="_gaq.push(['_trackEvent', 'Downloads', 'iis.xml']);">here</a>.</p>

<p><b>Patching for different environments</b><br />
Some configuration settings may be specific to a particular environment in which you you wish to run your application. For example, the <code>ConstructorString</code> property of an object may be different if you're running your application on a development server rather than a production server. The scripts allow you to save complete applications and also application patches. A patch is simply a subset of the XML required to create the entire application, it can be as many or as few settings as you like and can be applied after the main application has been recreated to allow you to tailor the application for a particular environment. At present you have to create the patch files manually using a tool like the <a href="http://msdn.microsoft.com/xml/notepad/download.asp">XML notepad</a>, but I intend to produce some scripts which will make this process easier [I never did - Len, 2005]. An application patch element is different only in that the element name is <code>ApplicationPatch</code> rather than <code>Application</code> and it has a <code>Patch</code> attribute that is used to allow a file to contain different patches for the same application. An example can be found <a href="http://www.lenholgate.com/zips/app-patch.xml" onclick="_gaq.push(['_trackEvent', 'Downloads', 'app-patch.xml']);">here</a>.</p>

<p>Run each script without any parameters for a help screen listing available options.</p>

<p><b>Download</b><br />
The following scripts are written in VB script and have been tested on Windows 2000.<br />
<a href="http://www.lenholgate.com/zips/COM+DumpApp.vbs" onclick="_gaq.push(['_trackEvent', 'Downloads', 'COM+DumpApp.vbs']);">COM+DumpApp.vbs</a><br />
<a href="http://www.lenholgate.com/zips/COM+InstallApp.vbs" onclick="_gaq.push(['_trackEvent', 'Downloads', 'COM+InstallApp.vbs']);">COM+InstallApp.vbs</a></p>

<p><b>Revision history</b><br />
</p><ul><li>11th June 2000 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>.</li>
<li>12th June 2000 - A few bug fixes. </li>
<li>15th June 2000 - Bug fix: save component properties before configuring component, interface and method roles. If the "ComponentAccessChecksEnabled" property isn't set to true and the component saved before configuring roles then the role configuration is silently ignored. </li>
<li>29th June 2000 - New functionality. You can now create patch files that include a prompt and a default value. If a prompt is present then a dialog box is displayed and the user is prompted for the value that will be patched - useful for password requests, etc. You can also add a prompt to each application and application patch so that the user can be given some information about what they're installing. </li>
<li>27th September 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li>
</ul><p></p>]]>
    </content>
</entry>

<entry>
    <title>OLEDB - Disconnected Recordsets</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2000/05/oledb---disconnected-recordsets.html" />
    <id>tag:www.socketframework.com,2000:/blog//12.128</id>

    <published>2000-05-21T00:00:01Z</published>
    <updated>2010-12-22T13:22:12Z</updated>

    <summary>If you are going to use the client cursor engine then often it&apos;s a good idea to disconnect your recordset......</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="OLEDB" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Reprints" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Source Code" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Way back" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        If you are going to use the client cursor engine then often it&apos;s a good idea to disconnect your recordset...
        <![CDATA[<p><b>Disconnected Recordsets</b><br />
The normal procedure for using the Client Cursor Engine is to open your recordset with client side cursors and then disconnect it from the data source. This causes the CCE to pull all of your data out of the data source and effectively marshal it by value to your client. Though the code available so far allows the use of the CCE there's no way to disconnect the recordset from your simple data object. We'll address that issue now.</p>

<p><b>What happens when we disconnect?</b><br />
When you disconnect a recordset by setting the active connection to nothing what happens inside your provider is that the <code>Cancel</code> method is called on your command object. In fact, the <code>Cancel</code> method is always called when the consumer is done with the underlying data source. Thus, we can intercept the call to <code>Cancel</code> and use it as a trigger to release our connection to the data object. To be able to do this we need to be able to communicate between the <code>Command</code> object and the rowset that it creates. To achieve this communication we can add a new interface to our proxy rowset:</p>
<pre class="brush: cpp gutter: false">interface IDisconnectableRowset : IUnknown
{
   HRESULT DisconnectRowset();
};</pre>
<p>We can then query for this interface from our <code>Command</code> object as the last thing we do inside of the <code>Execute</code> method.</p>
<pre class="brush: cpp gutter: false">if (SUCCEEDED(hr))
{
   hr = pGetAsRowset-&gt;GetAsRowset(
      pOurUnknown, 
      pUnkOuter, 
      riid, 
      pcRowsAffected, 
      ppRowset);
           
   pOurUnknown-&gt;Release();
}
  
if (SUCCEEDED(hr))
{
   hr = (*ppRowset)-&gt;QueryInterface(
      IID_IDisconnectableRowset, 
      (void**)&amp;m_pRowset);
}
  
pGetAsRowset-&gt;Release();</pre>
<p>The <code>Command</code> object now has a way to tell the rowset that it has been disconnected, so we can add an implementation of <code>ICommand::Cancel()</code> that does that.</p>
<pre class="brush: cpp gutter: false">HRESULT CConversionProviderCommand::Cancel()
{
   HRESULT hr = ICommandTextImpl<cconversionprovidercommand>::Cancel();
  
   ATLTRACE2(atlTraceDBProvider, 0, "CConversionProviderCommand::Cancel\n");
  
   if (SUCCEEDED(hr))
   {
      if (m_pRowset)
      {
         hr = m_pRowset-&gt;DisconnectRowset();
         m_pRowset-&gt;Release();
         m_pRowset = 0;
      }
   }
  
   return hr;
}</cconversionprovidercommand></pre>
<p>First calling our base class to see if we can be cancelled and if so performing the disconnection on the rowset object.</p>

<p><b>What does the Rowset do?</b><br />
I chose to make all rowsets derived from <code>CProxyRowsetImpl</code> be disconnectable, so that's where the implementation of <code>IDisconnectableRowset::DisconnectRowset()</code> lives. It's fairly simple, it just causes the rowset to release the data object that it's a proxy rowset for. If nobody else has a reference to the data object at that point then it will cease to exist, that seems pretty disconnected to me...</p>
<pre class="brush: cpp gutter: false">template &lt;
   class DataClass, 
   class T, 
   class CreatorClass, 
   class Storage, 
   class ArrayType, 
   class RowClass, 
   class RowsetInterface&gt;
HRESULT CProxyRowsetImpl&lt;
   DataClass, 
   T, 
   CreatorClass, 
   Storage, 
   ArrayType, 
   RowClass, 
   RowsetInterface&gt;::DisconnectRowset()
{
   ObjectLock lock(this);
  
   if (m_pDataObject)
   {
      m_pDataObject = 0;
   }
  
   if (m_pUnkDataObject)
   {
      m_pUnkDataObject-&gt;Release();
      m_pUnkDataObject = 0;
   }
  
   return S_OK;
}</pre>
<p>That's all there is to it. We now have a disconnectable rowset.</p>

<p><b>Download</b><br />
The following source built using Visual Studio 6.0 SP3. Using the July 2000 edition of the Platform SDK. If you don't have the Platform SDK installed then you may find that the compile will fail looking for "msado15.h". You can fix this problem by creating a file of that name that includes "adoint.h".<br />
If your system drive isn't <code>D:\</code> then you'll have to change the #import statements in IGetAsADORecordsetImpl.h and IGetAsADORecordset.cpp.<br />
</p><ul><li><a href="http://www.lenholgate.com/zips/SimpleDataObject-7.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'SimpleDataObject-7.zip']);">Simple Data Object with Disconnected Recordset support</a></li>
<li><a href="http://www.lenholgate.com/zips/C++GridTest.zip" onclick="_gaq.push(['_trackEvent', 'Downloads', 'C++GridTest.zip']);">C++ client example</a></li>
<li><a href="http://www.janusys.com">Get a trial version of the Janus GridEx that is used in the test harness above</a></li></ul><br />
<b>Revision history</b><br />
<ul><li>21st May 2000 - Initial revision at <a href="http://www.jetbyte.com">www.jetbyte.com</a>.</li>
<li>2nd October 2000 - Fixed some build configuration errors. Thanks to Charles Finley for reporting these. </li>
<li>9th February 2001 - Added the C++ client example.</li>
<li>19th October 2005 - reprinted at <a href="http://www.lenholgate.com">www.lenholgate.com</a>.</li></ul>
<b>Other articles in the series</b><br />
<ul><li><a href="http://www.lenholgate.com/archives/000541.html">Objects via ADO</a> - ADO seems to be the ideal way to expose tabular data from your own COM objects and the ATL OLE DB Provider templates can help!</li>
<li><a href="http://www.lenholgate.com/archives/000542.html">Custom Rowsets</a> - The ATL OLE DB Provider templates appear to rely on the fact that your data is kept in a simple array, but that's not really the case at all!</li>
<li><a href="http://www.lenholgate.com/archives/000543.html">IRowsetLocate and Bookmarks</a> - Adding bookmark functionality is relatively easy and it enables our ADO recordset to be used with a greater number of data bound controls.</li>
<li><a href="http://www.lenholgate.com/archives/000544.html">Updating data through an ADO recordset</a> - The ATL OLE DB Provider templates only seem to support read-only rowsets, and making them support updating of data isn't as easy as you'd expect!</li>
<li><a href="http://www.lenholgate.com/archives/000545.html">Client Cursor Engine updates</a> - Making the ADO Client Cursor Engine believe that your rowset is updateable involves jumping through a few extra hoops...</li>
<li><b>Disconnected Recordsets</b> - If you are going to use the client cursor engine then often it's a good idea to disconnect your recordset...</li></ul><p></p>]]>
    </content>
</entry>

</feed>


