The great source code shuffle...

| 10 Comments

I'm in the process of moving my source code from a CVS repository to Subversion. The main reason for the move is so that I can move the repository onto one of my NAS devices so that it can be backed up automatically. I have a spare NAS due to the fact that I had a power supply fail and the fastest way to get up and running again was to buy a new one rather than waiting for the replacement PSU... Anyway, the development NAS has SVN running on it which means that I now have a RAIDed, UPS protected SVN server which gets backed up nightly to my 'off site' NAS via rsync...

I've been meaning to move from CVS for a while but the fact that I made heavy use of CVS 'modules' meant that it was complicated to move to SVN as there was no suitable replacement for 'modules'. I eventually decided to rework my source control usage to fit with SVN and to try and iron out a few of the wrinkles that have developed over the past 6 or so years that I've been using CVS.

The most notable improvement so far comes at the cost of duplicating some source... Many of my projects use source from my 'JetByteTools' libraries. In CVS I used a single repository for all source and relied on tags and branches to make sure that each project got the right version of shared code. This worked OK but was complicated to manage at times and the tools libs ended up with lots and lots of tags and branches. Every now and then I'd want to upgrade a project to use the latest version of the tools libs and this was complex to achieve. The CVS situation had evolved over several years and, to be honest, was far from ideal. One of the main problems was that it wasn't easy to remove projects when they were no longer required; old client code is often moved out to an external off-site archive and this can be complex to achieve if everything is in the same repository. So, anyway, when I started my move to SVN I decided to have a repository per project so that I could completely remove and archive old stuff without affecting work in progress (simply burn the repository to dvd and file it). To make sure that all repositories were self contained they contain copies of the tools libs that they need. Rather than simply branching within a repository I effectively branch into a new repository. So far this works well and merging to and from the 'main' tools library repository is no more complex than when everything was in one repository and on separate branches...

Once that was done I decided to clean up how The Server Framework's examples were stored. Way back when I first developed The Server Framework I decided that it was important that each example be buildable and packagable as a standalone thing. At the time I had half a dozen examples and one client might purchase features that required one or two different example servers to demonstrate. I now have around 8 different "packs" of examples, each of which contains several example servers... The problem with the original source tree design was that each example within a "pack" had its own copies of the shared tools library code. This was so that you could extract and zip a single directory to get a single example. Recent releases have included a script which copies a single set of tools lib source into the right places for the whole "pack", and the only thing that was keeping me from changing to a more useful structure was the momentum of my existing CVS source control system. Well, since I'm changing things anyway I decided to change the way the example "packs" work. Now they're slightly more fine grained, slightly more descriptive and each "pack" only includes a single copy of the code that's shared between all of the examples in the pack. I can still build 'single example' structures if I need to (and I haven't needed to for a long, long time) but for the usual use case things are easier. Each pack now has a script which can be run to pull all of required tools libraries from the repository into the right place and another script which will strip out all of the stuff that doesn't ship to clients (tests, mocks, etc). This seems to work well, reduces duplication, avoids the need for CVS modules and looks like it should scale better.

Finally I've taken a long hard look at the tools libraries and decided that some of them aren't worth keeping as separate libraries. The C++Tools lib was born back in 1997 when it was possible that we'd be developing cross platform code, at the time it made sense to keep stuff that wasn't tied to the windows platform in its own library. Now most of what's in there could be replaced by things from boost should I decide to add that as a dependency and the rest may as well live in the Win32Tools library. Likewise the RegistryTools library which was only separate because it was the subject of an article on CodeProject and which has missed out on several improvements because it wasn't in the Win32Tools lib (and so was easy to ignore).

The result of all this reshuffling wont be seen in The Server Framework until release 6 which is due near the end of Q1 2009. Due to the library reorganisation it's likely that there will need to be some header file and namespace usage changes in client code, but hopefully these will be minimal.

10 Comments

Have you looked at svn:externals?

Inside you example example directory you can use svn propset svn:externals to point to a path in another repository (eg. svn://.../tools/tags/1.0). Then "svn update" will automatically check it out and update it when you run "svn update" in the example directory

Thanks, I'll take a look. I was aware of it but was put off by the fact that externals weren't somehow linked to tags and branches (so could get out of sync) but, of course, nor are my scripts so the same problems apply.

Steady on the externals! They're not versioned along with the rest of the repository (or at least they never used to be).

So if you checked out an old revision and used externals, you'll won't necessarily get what you expect.

You could just check in a file that lists what you want each time and checkout that script and run it instead.

E.g.
svn co -N %SVNROOT%/trunk work
cd work
svn up admin
admin\get_the_rest.bat

and get_the_rest.bat has

svn up project1
svn up project2

etc.

I've already tried the 'script' route and that works. I'm looking at the externals now, it works and is slightly cleaner for general checkout and updates but I havent yet tried branching (and changing the externals on the branch). If that doesnt work then they're useless to me.

Barry, I don't know about old versions, but in Subversion 1.5.4 svn:externals ARE versioned. I justed tested it.

I agree that this is a crucial feature. The manual suggests that you use "peg" revision (ie. write revision numbers in the externals)

From http://svnbook.red-bean.com/en/1.5/svn.advanced.externals.html

"You should seriously consider using explicit revision numbers in all of your externals definitions. Doing so means that you get to decide when to pull down a different snapshot of external information, and exactly which snapshot to pull. Besides avoiding the surprise of getting changes to third-party repositories that you might not have any control over, using explicit revision numbers also means that as you backdate your working copy to a previous revision, your externals definitions will also revert to the way they looked in that previous revision, which in turn means that the external working copies will be updated to match the way they looked back when your repository was at that previous revision. For software projects, this could be the difference between a successful and a failed build of an older snapshot of your complex codebase."

Note that this backdating relies on svn:externals being versioned.

Let us know how it works out, Len.


Regards
Vagn Johansen

Hello,
The port I get is always 0, so the connect() does not work. I appreciate if you can help to solve the problem. The code is below:

//#include "stdafx.h"
#include
#include
#include
#include
// BA
#pragma comment(lib, "ws2_32.lib")

DEFINE_GUID (SAMPLE_UUID, 0x31b44148, 0x041f, 0x42f5, 0x8e, \
0x73, 0x18, 0x6d, 0x5a, 0x47, 0x9f, 0xc9);

int SDPGetPort(const char *addr, LPGUID guid) {
int port=0;
HANDLE h;
WSAQUERYSET *qs;
DWORD flags=0;
DWORD qs_len;
bool done;

qs_len = sizeof(WSAQUERYSET);
qs = (WSAQUERYSET*) malloc(qs_len);
ZeroMemory(qs, qs_len);
qs->dwSize = sizeof (WSAQUERYSET);
qs->lpServiceClassId=guid;
qs->dwNameSpace=NS_BTH;
qs->dwNumberOfCsAddrs=0;
qs->lpszContext = (LPSTR) addr;
flags = LUP_FLUSHCACHE | LUP_RETURN_ADDR;

if (SOCKET_ERROR == WSALookupServiceBegin(qs, flags, &h)) {
ExitProcess(2);
}

done = false;
while (!done) {
if (SOCKET_ERROR == WSALookupServiceNext(h, flags, &qs_len, qs)) {
int error = WSAGetLastError();
if (error == WSAEFAULT) { // 10014L
free(qs);
qs=(WSAQUERYSET*) malloc(qs_len);
} else if (error == WSA_E_NO_MORE) { // 10110L
done = true;
} else {
printf("WSALookupServiceNext() failed. error = [%d]\n", error);
ExitProcess(2);
}
} else {
SOCKADDR_BTH *sa = (SOCKADDR_BTH*) qs->lpcsaBuffer->RemoteAddr.lpSockaddr;
port = sa->port;
}
}
free(qs);

WSALookupServiceEnd(h);

return port;
}

// int _tmain(int argc, _TCHAR* argv[]) {
int main(int argc, char* argv[]) {
SOCKET sock;
SOCKADDR_BTH sa={ 0 };
int sa_len = sizeof (sa);

// initialize Windows sockets
WORD wVersionRequested;
WSADATA wsaData;
wVersionRequested = MAKEWORD(2,0);
if (WSAStartup(wVersionRequested, &wsaData) != 0) {
ExitProcess(2);
}

// Parse the specified Bletooth address
if(argc \n" \
"\n addr must be in the form (XX:XX:XX:XX:XX:XX)");
ExitProcess(2);
}
if (SOCKET_ERROR == WSAStringToAddress(argv[1], AF_BTH, NULL, (LPSOCKADDR) &sa, &sa_len)) {
printf("Debug: WSAStringToAddress() failed.\n");
ExitProcess(2);
}

// query it for the right port

// create the socket
sock = socket(AF_BTH, SOCK_STREAM, BTHPROTO_RFCOMM);
if (SOCKET_ERROR == sock) {
printf("Debug: socket() failed.\n");
ExitProcess(2);
} else {
printf("Debug: socket() success!\n");
}

// fill in the rest of the SOCKADDR_BTH struct
*********** THE FOLLOWING ALWAYS RETURNS PORT AS 0 ****************************
do {
sa.port = SDPGetPort(argv[1], (LPGUID) &SAMPLE_UUID);
printf("Debug: Port = %d\n", sa.port);
} while (sa.port == 0);

if (SOCKET_ERROR == connect(sock, (LPSOCKADDR) &sa, sa_len)) {
printf("connect() failed. error = [%d]\n", WSAGetLastError());
ExitProcess(2);
} else {
printf("Debug: connect() success!\n");
}

send(sock, "hello!", 6, 0);

closesocket(sock);

return 0;
} // main()

One of the most important things you learn as a programmer is how to ask the right questions. Your comment demonstrates how NOT to do this in SO many ways.

Firstly why did you pick this entry to post your question to? Why not post it on the appropriate bluetooth entry?

Secondly if you have debugged the code at all you'd probably be able to ask a more concise question; such as why does blah always fail (I assume that one of the API calls is failing, or returning data that you think is 'bad'?), rather than expecting ME to debug your code and then fix your bug. I can quote you a price to look at this for you if you really want me to.

Vagn,

Thanks, yes I found this out later after talking with Len offline. We use v.1.4 where externals weren't versioned, but they are now, so good news!

I think it's worth testing those externals on 1.4, I have 1.4.2 installed on my NAS and it DOES version externals (unless it's due to the 1.5 client that I'm using?)

You know what they say about assumptions :) We assumed that as it wasn't mentioned in the documentation (svn-book) that the externals weren't versioned. The client is independent of the server.

Leave a comment