Hello,
I have been doing some experiment with MPEG4IP (http://www.mpeg4ip.net/). MPEG4IP comes with bunch of related codec (both encoder and decoder... duh) and a higher level "player" program called mp4player with which you can view the movie and audio read from disk and send through the network.
I hacked the player software so that it writes the decoded audio and video data in buffers, instead of dumping it to the SDL audio and video library. The buffers are actually Squeak's SoundBuffer and Bitmap; you can use the data in Squeak. I haven't tested it well, but it should work with RTP or HTTP streamed video. Only non-standard library it uses is the multi-thread library of SDL, so it should be reasonably portable among major platforms.
I modified (or I think I modified) it so that it can handle multiple concurrent content streams at once. It is vital to make up a possible application like multi-person video conferencing system.
I haven't gone as far as I wanted to, though. The limitations are:
* The timing and sync is not working very nicely. (Especially, the order of video frames is often wrong.)
* The handling of concurrent/multiple streams is not tested well.
* Encoder part hasn't been worked at all.
* etc., etc.
The problem I had is the debugging a multi-threaded, realtime program written in C++. The program depends on the libraries that are about one million lines of C++ code in total, so this was logistically cumbersome to look at.
The reason I hacked mp4player was to avoid reimplementing different stream protocols like RTP and the architecture of pluggable codec plugins. But with hindsight, I could have done a better job if I just use the codecs as Squeak plugins and write the software layers above them in Squeak...
Anyway, for those who might be interested in it, I put the code on the web at:
http://www.is.titech.ac.jp/~ohshima/squeak/MPEG4IP/
First try the Windows version of binary "squeak_gui_test.exe" in
http://www.is.titech.ac.jp/~ohshima/squeak/MPEG4IP/mpeg4ip-squeak-dll-050103...
Start the program, and type a .mp4 file name (you can find bunch at http://www.archive.org/movies/movies.php) in the box, and press the "Start" and "Display" button in this order. It should show the video data on screen and write the sound data to a file.
Then, to compile the program, get mpeg4ip-1.2 source from the MPEG4IP sourceforge site, and apply the patch in
http://www.is.titech.ac.jp/~ohshima/squeak/MPEG4IP/mpeg4ip-squeak-050103.tar...
and open the player60.sln from Visual Studio. Build them in the "Release" configuration and you'll get the dlls and executable.
To do stuff from Squeak, get
http://www.is.titech.ac.jp/~ohshima/squeak/MPEG4IP/MPEG4IPClientPlugin-05010...
and file in the .st file into a recent Squeak image and evaluate:
m _ MPEG4IPClientMorph new openInWorld. m startClientOn: 'air64.mp4'.
I need to focus on the other stuff, but if somebody move forward from here, or take "the other path" I mentioned, it would be really great.
Thank you!
-- Yoshiki
squeak-dev@lists.squeakfoundation.org