On Sat, 26 Feb 2005 22:05:02 +0100, Avi Bryant avi.bryant@gmail.com wrote:
Yep, this sounds very cool. Looking forward to the next post from Cees saying "ok, so I went ahead and implemented this, here it is..."
Ok, so I went ahead and implemented this, here it is...
(from the completely-untested-but-highly-interesting-code-dept.)
Avi helped me a bit on IRC, so I could hash out a skeleton implementaton of TPMCRepository, a Monticello rep that sits on my p2p networking code. Before you go off and download it, note that:
a) it is completely untested - especially the front-end bits; b) it is completely and utterly insecure in that you don't know where a version comes from; c) it will announce the complete contents of your MC package cache to the network, including that secret code normally only available under an NDA signed with blood that you're writing for your next dotcom startup.
Actually, point a) is the best security feature at the moment, because chances are good that the software will not be able to do any harm ;). A next version will sport private key signing of packages that originate from your box and some yet-to-be-invented mechanism of distributing the public keys for signature checking out-of-band.
Anyway, what happens if you install the code: - MCCacheRepository gets a couple of overrides to announce its stuff to the network. The p2p code has presence code in it which tries to disseminate this information over the network in what I hope is a relatively efficient manner (there's a clear time/bandwidth trade off here, of course). - Announcements will be renewed every 24 hours as long as the file stays around; - TPMCRepository, if you add it, will read the local copy of these announcements (which hopefully gets stuffed with refreshments fromt the network all the time) and show these as the repository contents. - When you attempt to open a file, it'll ask the nodes that announced it, in turn, to send the file to it. If that succeeds, MC gets to do its thing. The file is added to the MCCacheRepository so that the next time around you'll have it locally (plus the side effect is that your node will now start announcing the file).
The goal of this thing is to see how large amounts of presence information can be dealt with. Most machines these days should be capable enough to handle very large sets of metadata pointers (MC presence info is just a filename and a originating peer UUID, so you should be able to keep a million of them around), but that will require something more scalable than the trivial implementations I used in the AardWorks-Gossip package. Maybe using the large collections will be enough, maybe some persistence is needed. Bandwidth usage is probably a matter of tuning rather than radical changes in the current approach but that's more a gut feeling than anything else.
If you want to look at it, grab http://tai42.xs4all.nl/~cg/mc/Tric-P2P-CdG.4.mcz and check the overrides in MCCacheRepository and the single class in Tric-P2P-MC.
If you want to play with it, fine. But send me fixes, not whines that it doesn't work :-P
Oh - if you have old, like pre-Friday, p2p code in your image, make sure to clean up any AWGPeer that is still running; I changed most timeout values from integers to durations, and that messes up running peers.
And BTW - if you want to help out, if nothing else, pull the code into a scratch image, do "AWGPeer newOnDefaultPort", iconize it, and let it simmer. The network currently is 2 nodes big, a bit more would be fun (especially nodes not behind NAT, they are a tremendous help in bootstrapping the network and acting as stable reference points).
Exercise for the reader: at the presence code to the chat app so we can take #squeak into the p2p age :)
Bedtime,
Cees