Hi,
In the MC rep at http://kilana.unibe.ch:8888/DGV there's a version of AardWorks-Gossip that we just tested with a couple of people on IRC. The code can form spontaneous networks and is able to traverse multiple NAT's. So what is there actually works, at least if you do not put your expectations of 'works' too high ;). There's a reasonable chance that actually trying to exchange data with other peers through the AWGPeer interface to the network will work, but you'd be the first to try.
The next step will be a demo app. I am thinking of a chat application because it has a nice mix of searching and direct-routing. The idea is that you get a button or world menu entry or whatever that says you're available for chat. Registration is of course under your developer initials :). Then, when something wants to chat with CdG, the p2p network hunts for the location of CdG, and starts a chat session.
Sounds neat and doable and the correct scope for a test app. But if anyone has a better idea for a killer app, shout.
(other idea: point to a local MC directory and declare that a publicly readable directory on the network. Presto, your code can now be found by package name by anyone on the network. V2 will add caching by intermediate nodes ;))
For those who want to sniff around, I'll leave a test server running on a publicly accessible box until it crashes or runs out of memory or goes off to make coffee. After loading the latest package from MC, do:
| peer | peer _ AWGPeer newOnUDPPort: 16rCA01. peer addPeer: UUID new host: (NetNameResolver addressForName: 'uk-1.aardworks.nl') port: 16rCA99. peer explore
This should connect you to the bootstrap peer (note that this is just *a* introducer - you need to start somewhere but when the network is larger and especially when someone cobbles up a mechanism to export peer lists through out-of-band mechanisms like HTTP it can go away). If it is still running, root->peerId will hold your dynamically assigned peer id. root->peerList->entries should fill up with up to 30 addresses of other peers, most of them with very low hopCount.
There's documentation in the code in classes that are named after their packages. If you itch to play with this, look at the sending-receiving protocol of AWGPeer. That is normally the only protocol you need for interaction with the network - it has methods to send data to an endpoint on a certain peer, and to register code blocks that are triggered when data comes in for a certain endpoint on the local peer (endpoints are very much like TCP or UDP ports). Of course, this currently all operates with ByteArrays, a neat higher protocol layer is one of the things next on the to-do list.
Happy hacking,
Cees
On Tue, 22 Feb 2005 01:31:03 +0100, I wrote:
The next step will be a demo app.
Well, it's *almost* there. Just needs a user interface. But the most important bit of the proof-of-concept is in the domain code, which is an application of the new flood searching stuff in the AardWorks-Gossip package.
http://kilana.unibe.ch:8888/DGV is the MC repository, you want the latest incarnation of 'Tric-P2P Chat' in there. If you have a AWGPeer running per the instructions from yesterday, you can now do
TPCNode onPeer: <your AWGPeer>
My UI still is the explorer ;).
Stuff that works:
TPCNode>>availableForChat - makes yourself available (thus findable) TPCNode>>findUser: - finds the peer/endPoint where the user is waiting for chat requests
what needs to be done on findUser: finding something is replying probably with a new endpoint and opening a chat window between these two endpoints. Anyway, that bit of the software needs some overhaul, I've been concentrating on everything behind #findUser:
wrt code reuse: is there a reasonably clean 1:1 chat morph somewhere? Not the eToys stuff, it looks too hardwired to their own chat protocol (and no, I ain't gonna refactor it ;)).
Feedback requested: it looks that responsibilities are nicely divided, except for the code in TPCNode>>receivedSearch:from: - comments on that code much appreciated.
Hopefully another update tomorrow...
On Wed, 23 Feb 2005 01:02:48 +0100, I wrote:
On Tue, 22 Feb 2005 01:31:03 +0100, I wrote:
The next step will be a demo app.
Well, it's *almost* there. Just needs a user interface.
Gee. I'm good at keeping promises this week.
http://kilana.unibe.ch:8888/DGV
now sports a version of Tric-P2P Chat that comes with a user interface. I just tested it and it seems to work. I also wrote it that it *should* exercise most of the code below it - reliable and unreliable datagrams, flood searching, static and dynamic endpoint creation, etcetera.
To start it, download the package (it'll fetch some prerequisites, including ToolBuilder and the Morphic implementation of ToolBuilder - if you want the UI to work under Tweak or MVC, fetch the corresponding packages from http://kilana.unibe.ch:8888/ToolBuilder yourself), and type the magic incantation
TPCNode open
If you have an instance of AWGPeer running in your image, it'll be used, otherwise you are asked whether you want to start one (you do, otherwise the thing will refuse to start, but I thought it polite behaviour nevertheless). A UI is opened with a red 'Not Available' button - press it to make it green and show your developer initials; this means you're available for chat. The text field besides can be used to enter someone's developer initials (try 'CdG' during waking hours ;)) and on hitting enter, a flood search will be initiated to find someone that's available with these initials. If you get a positive response, a chat window will be opened, you can type in the lower pane and see the log in the upper pane. If someone searches for you, gets a chat window, and starts to type, you will get a notifier asking you whether you want to chat back. If yes, a chat window is opened, if no, the source gets a message to the effect that you declined.
Usability Issues I Don't Know How To Solve (not without a lot of digging): - The logging windows don't scroll automagically. Logging sucks anyway with a text field, but it seems all that ToolBuilder has to offer at the moment; - In the chat window, after opening it the text input field below should have the focus; - The fact that I need a fat paragraph to describe how to use such a trivial app is a testament to the quality of the UI ;)
As usual, feedback welcome. Now that we have something usable, anyone interested in further work? Some generic p2p issues: - Seed lists: creation, out-of-band distribution. There's no other way to get onto the network, I fear; - Bandwidth management/monitoring; - Monitoring of other stuff, including using the p2p net to retrieve data on the net's structure for analysis; - etcetera.
Ok, the P2P chat thing works, a nice UI is left as an exercise for the interested reader, on to a next proof-of-concept.
Without doing formal coverage analysis, my gut feeling is that the P2P chat application hits most of the P2P framework. It looks like it works, so it's time to move to more interesting stuff that builds on this.
So, the next project is going to be hard. Really hard. Well... compared with the stuff I cobbled up so far. I've declared myself too stupid to do it all by myself (or too lazy, you choose) so I need your help.
The idea is to write an MCRepository backend on top of this. You create a MCP2PRepository, point to a (shadow/cache) file storage area, and from that point on everything you put there is accessible by everyone else who has added an MCP2PRepository to his/her image.
'accessible' is one thing, but managing flood searches, ensuring timely propagation, caching on intermediate nodes(?), reliable transfer of larger datagrams over the network (it *should* work in theory, right now...), a workable interface that accepts that opening such a repository will involve a lot of asynchronous updates, ...
You see, enough juicy stuff to sink your teeth into :)
I'm off to bed now. If you're interested, let it sink in, and start harassing me with questions on the current architecture if you need to have more info to solve these little puzzles.
(did I mention authentication etcetera?)
G'night,
Cees
Cees, very cool idea.
I'm interested in something a bit more modest - using your p2p network to broadcast Monticello notifications. By "notifications", I just mean that after posting a .mcz file to a repository, the Monticello instance would send out the URL for that file out to all the nodes.
I don't suppose that would test your framework very much, but it seems like it would be very useful, at least to folks using FTP and HTTP repos. And access control would come for free via the magic of delegation... :)
Hi Brent!
Brent Vukmer brent.vukmer@gmail.com wrote:
Cees, very cool idea.
I'm interested in something a bit more modest - using your p2p network to broadcast Monticello notifications. By "notifications", I just mean that after posting a .mcz file to a repository, the Monticello instance would send out the URL for that file out to all the nodes.
I think you should be more general and go for my idea I posted about way back about "developer events". So we define a few different "developer event types" that people can emit, and you of course decide yourself which event types you want to emit.
And you also define what kind of events you want to subscribe to - this Monticello release event type being one kind.
Such a system will open up endless cool things. Like "someone modified my class" event, or "someone just installed my package from SM" event, etc. Then we can start modifying our tools with visual real time feedback etc.
regards, Göran
I wonder if you could marry BitTorrent to the system. Use that protocol and framework (ported to Smalltalk of course) for handling all the actual seeding, transfer, etc. and use the MCP2PRepository system to act as the torrent trackers. Basically, anyone 2 or more people running MCP2PRepository together become one "tracker", sharing their torrent information to all other interested parties. When someone wants to get a package from the repository the system generates the required torrent info and then starts up the SqueakTorrent subsystem to do the actual work.
Caveat: I am new to Squeak and know nothing about Monticello. Just seems like you could leverage a lot of work and understanding from the BT world to get you over the issues outlined below.
joe
On Feb 23, 2005, at 4:13 PM, Cees de Groot wrote:
Ok, the P2P chat thing works, a nice UI is left as an exercise for the interested reader, on to a next proof-of-concept.
Without doing formal coverage analysis, my gut feeling is that the P2P chat application hits most of the P2P framework. It looks like it works, so it's time to move to more interesting stuff that builds on this.
So, the next project is going to be hard. Really hard. Well... compared with the stuff I cobbled up so far. I've declared myself too stupid to do it all by myself (or too lazy, you choose) so I need your help.
The idea is to write an MCRepository backend on top of this. You create a MCP2PRepository, point to a (shadow/cache) file storage area, and from that point on everything you put there is accessible by everyone else who has added an MCP2PRepository to his/her image.
'accessible' is one thing, but managing flood searches, ensuring timely propagation, caching on intermediate nodes(?), reliable transfer of larger datagrams over the network (it *should* work in theory, right now...), a workable interface that accepts that opening such a repository will involve a lot of asynchronous updates, ...
You see, enough juicy stuff to sink your teeth into :)
I'm off to bed now. If you're interested, let it sink in, and start harassing me with questions on the current architecture if you need to have more info to solve these little puzzles.
(did I mention authentication etcetera?)
G'night,
Cees
On Wed, 23 Feb 2005 17:58:28 -0800, Joseph Jones darkdescendant@mac.com wrote:
I wonder if you could marry BitTorrent to the system.
Well, try it ;P However, BT is made for LARGE files, and MC files typically aren't that large. My P2P system sits on a system that allows for reliable transmission of datagrams of up to 6Mb, I think, plenty enough for most MC files.
Goran: developer events is only part of the picture, of course. Getting MC files to you in reasonable time is another. A shared global MC rep would mean you'd never have to copy a dependent MC file to a repository, for example - you just get it from the network. But yeah, developer events would be cool and simple to do with the modern systemchangenotification stuff. I might hack them in tonight :)
On Thu, 24 Feb 2005 01:13:39 +0100, Cees de Groot cg@cdegroot.com wrote:
The idea is to write an MCRepository backend on top of this. You create a MCP2PRepository, point to a (shadow/cache) file storage area, and from that point on everything you put there is accessible by everyone else who has added an MCP2PRepository to his/her image.
Yep, this sounds very cool. Looking forward to the next post from Cees saying "ok, so I went ahead and implemented this, here it is..."
Cees, if you do come up with some spike solution to this and want to test it with a few people, let me know. I can play the "medium latency: same country, different network" role. (Note how I'm very carefully *not* offering to implement any of it ;).
Avi
On Sat, 26 Feb 2005 22:05:02 +0100, Avi Bryant avi.bryant@gmail.com wrote:
Yep, this sounds very cool. Looking forward to the next post from Cees saying "ok, so I went ahead and implemented this, here it is..."
I'm almost there. I am currently testing a primitive Presence implementation so stuff can be announced as being available on the network. So you announce that a certain .mcz file is available, the bit about searching and retrieving it is then relatively straightforward. I'm probably going ugly on this and just hook the thing into the cache repository - it's a spike, after all.
(Note how I'm very carefully *not* offering to implement any of it ;).
Lazy guy, you ;-P
OBTW - the other thing I'm thinking about doing is Goran's idea of developer events. A simple publish/subscribe mechanism and hook into the new system change notification stuff, and when you press accept my image beeps or something.
On Sat, 26 Feb 2005 22:05:02 +0100, Avi Bryant avi.bryant@gmail.com wrote:
Yep, this sounds very cool. Looking forward to the next post from Cees saying "ok, so I went ahead and implemented this, here it is..."
Ok, so I went ahead and implemented this, here it is...
(from the completely-untested-but-highly-interesting-code-dept.)
Avi helped me a bit on IRC, so I could hash out a skeleton implementaton of TPMCRepository, a Monticello rep that sits on my p2p networking code. Before you go off and download it, note that:
a) it is completely untested - especially the front-end bits; b) it is completely and utterly insecure in that you don't know where a version comes from; c) it will announce the complete contents of your MC package cache to the network, including that secret code normally only available under an NDA signed with blood that you're writing for your next dotcom startup.
Actually, point a) is the best security feature at the moment, because chances are good that the software will not be able to do any harm ;). A next version will sport private key signing of packages that originate from your box and some yet-to-be-invented mechanism of distributing the public keys for signature checking out-of-band.
Anyway, what happens if you install the code: - MCCacheRepository gets a couple of overrides to announce its stuff to the network. The p2p code has presence code in it which tries to disseminate this information over the network in what I hope is a relatively efficient manner (there's a clear time/bandwidth trade off here, of course). - Announcements will be renewed every 24 hours as long as the file stays around; - TPMCRepository, if you add it, will read the local copy of these announcements (which hopefully gets stuffed with refreshments fromt the network all the time) and show these as the repository contents. - When you attempt to open a file, it'll ask the nodes that announced it, in turn, to send the file to it. If that succeeds, MC gets to do its thing. The file is added to the MCCacheRepository so that the next time around you'll have it locally (plus the side effect is that your node will now start announcing the file).
The goal of this thing is to see how large amounts of presence information can be dealt with. Most machines these days should be capable enough to handle very large sets of metadata pointers (MC presence info is just a filename and a originating peer UUID, so you should be able to keep a million of them around), but that will require something more scalable than the trivial implementations I used in the AardWorks-Gossip package. Maybe using the large collections will be enough, maybe some persistence is needed. Bandwidth usage is probably a matter of tuning rather than radical changes in the current approach but that's more a gut feeling than anything else.
If you want to look at it, grab http://tai42.xs4all.nl/~cg/mc/Tric-P2P-CdG.4.mcz and check the overrides in MCCacheRepository and the single class in Tric-P2P-MC.
If you want to play with it, fine. But send me fixes, not whines that it doesn't work :-P
Oh - if you have old, like pre-Friday, p2p code in your image, make sure to clean up any AWGPeer that is still running; I changed most timeout values from integers to durations, and that messes up running peers.
And BTW - if you want to help out, if nothing else, pull the code into a scratch image, do "AWGPeer newOnDefaultPort", iconize it, and let it simmer. The network currently is 2 nodes big, a bit more would be fun (especially nodes not behind NAT, they are a tremendous help in bootstrapping the network and acting as stable reference points).
Exercise for the reader: at the presence code to the chat app so we can take #squeak into the p2p age :)
Bedtime,
Cees
On Sun, 27 Feb 2005 03:32:51 +0100, Cees de Groot cg@cdegroot.com wrote:
If you want to look at it, grab http://tai42.xs4all.nl/~cg/mc/Tric-P2P-CdG.4.mcz and check the overrides in MCCacheRepository and the single class in Tric-P2P-MC.
If you grab http://tai42.xs4all.nl/~cg/mc/Tric-P2P-CdG.7.mcz instead, you'll get a version that I tested. Succesfully installed upgrades with the P2P rep, so it seems to work.
Cees de Groot wrote:
If you grab http://tai42.xs4all.nl/~cg/mc/Tric-P2P-CdG.7.mcz instead, you'll get a version that I tested. Succesfully installed upgrades with the P2P rep, so it seems to work.
I get:
$ wget http://tai42.xs4all.nl/~cg/mc/Tric-P2P-CdG.7.mcz --13:56:48-- http://tai42.xs4all.nl/%7Ecg/mc/Tric-P2P-CdG.7.mcz => `Tric-P2P-CdG.7.mcz' Resolving tai42.xs4all.nl... 80.126.80.66 Connecting to tai42.xs4all.nl[80.126.80.66]:80... connected. HTTP request sent, awaiting response... 403 Forbidden 13:56:48 ERROR 403: Forbidden.
On Sun, 27 Feb 2005 13:57:33 +0000, Tony Garnock-Jones tonyg@lshift.net wrote:
$ wget http://tai42.xs4all.nl/~cg/mc/Tric-P2P-CdG.7.mcz --13:56:48-- http://tai42.xs4all.nl/%7Ecg/mc/Tric-P2P-CdG.7.mcz => `Tric-P2P-CdG.7.mcz' Resolving tai42.xs4all.nl... 80.126.80.66 Connecting to tai42.xs4all.nl[80.126.80.66]:80... connected. HTTP request sent, awaiting response... 403 Forbidden 13:56:48 ERROR 403: Forbidden.
Sorry... Fixed
And as it seems to work, I'll stop using obsolete stuff like SqueakSource and Apache HTTP dirs for now ;). Next update is available on the network (very minor, but I wanted to go into dogfood mode just for the heck of it).
Cees de Groot wrote:
And as it seems to work, I'll stop using obsolete stuff like SqueakSource and Apache HTTP dirs for now ;). Next update is available on the network (very minor, but I wanted to go into dogfood mode just for the heck of it).
Very cool, Cees! I've just found the "+Repository" Tric P2P option. I've had a TPMCRepository running overnight from a Workspace, and wondered how to actually install things from it this morning.
After adding the Tric P2P repo to my browser, and clicking Open, I got a nice summary with a few packages and quite some (.mcz)s against each package. Clicking on the latest Tric-P2P package (Tric-P2P-CdG.12.mcz) paused the whole image for about a minute before resulting in an error, "sorry, couldn't find anything on the p2p net" (walkback pasted below).
Perhaps the timeout is too short? (Alternatively, from a user-feedback POV, it's already too long :-) ) Looking at the value of "hits" in "readStreamForFileNamed:do:", there are two separate entries.
Regards, Tony
28 February 2005 8:39:09 am
VM: Win32 - a SmalltalkImage Image: Squeak3.7 [latest update: #5989]
SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir J:\Squeak\Squeak3.7u1 Trusted Dir J:\Squeak\Squeak3.7u1\tonyg Untrusted Dir C:\My Squeak\tonyg
TPMCRepository(Object)>>error: Receiver: a TPMCRepository(Tric P2P) Arguments and temporary variables: aString: 'sorry, couldn''t find anything on the p2p net' Receiver's instance variables: creationTemplate: nil cache: a Dictionary() peer: an AWGPeer
TPMCRepository>>readStreamForFileNamed:do: Receiver: a TPMCRepository(Tric P2P) Arguments and temporary variables: aString: 'Tric-P2P-CdG.12.mcz' aBlock: [] in TPMCRepository(MCFileBasedRepository)>>versionReaderForFileNamed:...etc... presence: an AWGPresence hits: a Set(an AWGPresenceEntry(Tric-P2P-CdG.12.mcz@2e76ca5f-a5e9-4b90-8f14-7f5...etc... found: false each: an AWGPresenceEntry(Tric-P2P-CdG.12.mcz@b9883907-bbf4-b74b-8795-6c4188889...etc... Receiver's instance variables: creationTemplate: nil cache: a Dictionary() peer: an AWGPeer
TPMCRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: Receiver: a TPMCRepository(Tric P2P) Arguments and temporary variables: aString: 'Tric-P2P-CdG.12.mcz' aBlock: [] in TPMCRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: ...etc... s: nil class: nil Receiver's instance variables: creationTemplate: nil cache: a Dictionary() peer: an AWGPeer
TPMCRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: Receiver: a TPMCRepository(Tric P2P) Arguments and temporary variables: aString: 'Tric-P2P-CdG.12.mcz' r: nil Receiver's instance variables: creationTemplate: nil cache: a Dictionary() peer: an AWGPeer
--- The full stack --- TPMCRepository(Object)>>error: TPMCRepository>>readStreamForFileNamed:do: TPMCRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: TPMCRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [] in TPMCRepository(MCFileBasedRepository)>>versionFromFileNamed: {[self loadVersionFromFileNamed: aString]} [] in Dictionary>>at:ifAbsentPut: {[self at: key put: aBlock value]} Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: TPMCRepository(MCFileBasedRepository)>>versionFromFileNamed: MCFileRepositoryInspector>>versionSelection: PluggableListMorph>>changeModelSelection: PluggableListMorph>>mouseUp: PluggableListMorph(Morph)>>handleMouseUp: MouseButtonEvent>>sentTo: PluggableListMorph(Morph)>>handleEvent: PluggableListMorph(Morph)>>handleFocusEvent: [] in HandMorph>>sendFocusEvent:to:clear: {[ActiveHand := self. ActiveEvent := anEvent. result := focusHolder han...]} [] in PasteUpMorph>>becomeActiveDuring: {[aBlock value]} BlockContext>>on:do: PasteUpMorph>>becomeActiveDuring: HandMorph>>sendFocusEvent:to:clear: HandMorph>>sendEvent:focus:clear: HandMorph>>sendMouseEvent: HandMorph>>handleEvent: HandMorph>>processEvents [] in WorldState>>doOneCycleNowFor: {[:h | ActiveHand := h. h processEvents. capturingGesture := capturingGest...]} Array(SequenceableCollection)>>do: WorldState>>handsDo: WorldState>>doOneCycleNowFor: WorldState>>doOneCycleFor: PasteUpMorph>>doOneCycle [] in Project class>>spawnNewProcess {[[World doOneCycle. Processor yield. false] whileFalse. nil]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]}
On Mon, 28 Feb 2005 08:44:26 +0000, Tony Garnock-Jones tonyg@lshift.net wrote: Clicking on the latest Tric-P2P package (Tric-P2P-CdG.12.mcz)
paused the whole image for about a minute before resulting in an error, "sorry, couldn't find anything on the p2p net" (walkback pasted below).
Don't past walkbacks, fix them ;). Actually, I'm just mimicking MC's regular behavior here by throwing walkbacks instead of nice error dialogs. Bugger Avi about this, please ;) (or not, it's a developer tool, after all)
Perhaps the timeout is too short? (Alternatively, from a user-feedback POV, it's already too long :-) ) Looking at the value of "hits" in "readStreamForFileNamed:do:", there are two separate entries.
A problem right now is that the network is too unreliable because of its size. That makes me a bit wary of suggesting alternative ways to tackle this at this point in time.
Ramble: - For the user it would be better to send the retrieval request to all hits in parallel; for the network that wouldn't be as nice of course; - So maybe a two-stage protocol? Ask for a file handle, retrieve from the first host that responds?
Of course, the reason that I'm putting out this stuff is that I'd like to have some input on options like these - there are a lot of time/space/bandwidth tradeoffs to be made in p2p land...
Cees de Groot wrote:
Don't past walkbacks, fix them ;).
Mea culpa - you did say "But send me fixes, not whines that it doesn't work", after all :) (My excuse is that doesn't seem to be a codefix kind of thing)
Actually, I'm just mimicking MC's regular behavior here by throwing walkbacks instead of nice error dialogs. Bugger Avi about this, please ;) (or not, it's a developer tool, after all)
I don't mind that too much.
A problem right now is that the network is too unreliable because of its size. That makes me a bit wary of suggesting alternative ways to tackle this at this point in time.
Okay. I'll have to do a bit of digging to find out how messages are routed to the two hits - 30 seconds seems like a loooooong time, so I guess I need to learn more about how the p2p network works to understand why 30 seconds isn't practically infinity.
In fact I'll, just as an experiment, try setting it to 300 seconds and see what happens.
- So maybe a two-stage protocol? Ask for a file handle, retrieve from
the first host that responds?
I think that sounds like a good idea. Perhaps have a small number, say 5, of first-stage-handle-requests going simultaneously; as they time out, replace them with more from the pool of untried hits; and if one responds positively, cancel the other 4 and take the file from the offer.
Of course, the reason that I'm putting out this stuff is that I'd like to have some input on options like these - there are a lot of time/space/bandwidth tradeoffs to be made in p2p land...
Agreed :) How does bittorrent do its thing, I wonder? I did read the protocol description once many moons ago but it didn't stick, clearly...
Tony
On Mon, 28 Feb 2005 12:03:07 +0000, Tony Garnock-Jones tonyg@lshift.net wrote:
Okay. I'll have to do a bit of digging to find out how messages are routed to the two hits - 30 seconds seems like a loooooong time, so I guess I need to learn more about how the p2p network works to understand why 30 seconds isn't practically infinity.
It is. If the thing works, I found it much faster than SqueakSource in responsiveness and comparable to an average remote repository.
In fact I'll, just as an experiment, try setting it to 300 seconds and see what happens.
Nothing. I think the nodes that claimed to have the file you wanted simply weren't available. That's why network size matters here. And stable code, of course ;)
Agreed :) How does bittorrent do its thing, I wonder? I did read the protocol description once many moons ago but it didn't stick, clearly...
Bittorrent asks lots of peers for blocks, I think. With direct connections, there's no peer-to-peer routing of payload as far as I'm aware. Gnutella does p2p routing of "metadata" (searches and responses), but not of data - actual files are transported out-of-band with direct HTTP-ish connections. Something I'd like to postpone as long as possible ;) (I think for large files, say >>100k, things should be splitup in blocks and one could retrieve individual blocks).
Anyway, I think that the mod where you ask for a 'file handle' or 'transfer handle' to a peer first is probably the most sensible thing to change on the current implementation.
Cees de Groot wrote:
Nothing. I think the nodes that claimed to have the file you wanted simply weren't available. That's why network size matters here. And stable code, of course ;)
I see! How about, when a timeout happens, that hit is removed from the hit set? Does that seem sensible?
Tony
On Mon, 28 Feb 2005 13:18:21 +0000, Tony Garnock-Jones tonyg@lshift.net wrote:
I see! How about, when a timeout happens, that hit is removed from the hit set? Does that seem sensible?
You mean that it is marked as not being present anymore? Possible, but not really necessary. It'll time out in any case, and if it's just an ADSL connection going bang for a minute or two it's a bit heavyhanded to declare the host dead...
Anyway, that's probably advanced optimization. I'll first want to try to get the two-stage approach going.
On Feb 28, 2005, at 2:58 AM, Cees de Groot wrote:
Ramble:
- For the user it would be better to send the retrieval request to all
hits in parallel; for the network that wouldn't be as nice of course;
- So maybe a two-stage protocol? Ask for a file handle, retrieve from
the first host that responds?
Absolutely! This will be the simplest way to localize bandwidth and resource usage for peers that need the resources at a given time. That saves the overall network resources for the myriad of messages that will grow as more services are available. One of things about gnutella and other p2p networks out there, is the number of services available hasn't changed a whole lot - they are mostly being use for file sharing. So first we are putting MC repositories up, plus chat, then developer events, Pair programming browsers, Modules repositories, p2p Squeakmap, shared class /module documentation, BFAV on steroids, etc.... as you can see, the number of messages related to the just the services without the actual files being shipped around will grow dramatically as the network grows, so partitioning off things that are essentially one to one file transfers makes more sense to me.
Brian
Cees de Groot cg@cdegroot.com wrote:
And as it seems to work, I'll stop using obsolete stuff like SqueakSource and Apache HTTP dirs for now ;). Next update is available on the network (very minor, but I wanted to go into dogfood mode just for the heck of it).
Go go dog! :) And I look forward to that stab on "developer events" - I *really* think that would be super COOL to have in Squeak - it would be one of the first *really unique* features of Squeak as an "IDE" compared to the rest of the world.
regards, Göran
On Mon, 28 Feb 2005 10:21:41 +0100, goran.krampe@bluefish.se wrote:
Go go dog! :) And I look forward to that stab on "developer events" - I *really* think that would be super COOL to have in Squeak - it would be one of the first *really unique* features of Squeak as an "IDE" compared to the rest of the world.
Ok, let's make a deal: you describe a simple most useful developer event, including what should be done when one is received, and I code it up.
On Wed, 23 Feb 2005 23:49:30 +0100, Cees de Groot cg@cdegroot.com wrote:
If you're running this stuff, please update and restart. I transported my laptop to a different LAN with two nodes running on it, which unveiled some bugs in the peer list exchange code ;)
squeak-dev@lists.squeakfoundation.org
-
Avi Bryant -
Brent Vukmer -
Brian Brown -
Cees de Groot -
goran.krampe＠bluefish.se -
Joseph Jones -
Tony Garnock-Jones