I don't think that Apple created one preference interface for Mac OS X server and another for the standard Mac OS X distribution so it's probably not that.
The only thing I can think of is that, if your computer is sat behind a router (or Airport base-station) then the IP address you need to use to connect to the computer is that of the router; not the IP displayed at the bottom of the preferences. You also need to make sure that the appropriate ports are open, especially if you have a firewall?
I haven't really played with this feature since my laptop generally goes wherever I go and since I have Mac OS X standard I can't say for the server edition.
Hope this helps,
Mark.