We are very glad to announce the first beta release of SqueakDBX.
The aim of this project is to build an OpenDBX (http://www.linuxnetworks.de/doc/index.php/OpenDBX/ ) plugin which will allow users to perform relational database operations (DDL, DML and SQL) through a truly open source library.
This project has been selected as part of ESUG, SummerTalk 2008 (please, visit http://www.esug.org/Promotion/SummerTalk).
Through this project, the squeak community will hopefully be able to interact with major database engines such as Oracle and MSSQL besides open sources Postgresql and MySQL. Moreover, integrating this plugin with GLORP (http://www.glorp.org/), will allow us to generate a complete and open source solution to relational data base access.
We have been working in this project since january and we finally get our first beta release. This release does not have all the features we want it to have, just some of them, but we think is very usable.
The features for this release are:
Tested on 3.10 and Pharo. Support for Linux and OSX. Proved on windows (through MinGW), but some changes on OpenDBX are still needed (next version will have full compatibility). Tested on PostgreSQL, MySQL and Oracle. MS SQL Server, Firebird, Interbase, SQLite, SQLite3 and Sybase tests will be available as soon as possible. Transactional management. Automatic conversion of resultset columns (a String) into squeak types. Large objects (clob, blob, arrays, and so on) are not yet supported. Special OpenDBX options: multi-statments, compression, paged results. Automated database connection release on garbage collection (although manual disconnection is recommended ;-) Error handling
Good news: Lots of unit tests (82 right now, and more coming) The code is commented Full documentation at wiki There are some benchmarks done and it seems to be as fast as native driver There are some benchmarking (more needed), but shows we are reaching good performance (http://wiki.squeak.org/squeak/6063)
We think its important for you to know: We just test with MySQL 5, Postgres 8.3 and Oracle 10g XE. We just test with Ubuntu and OSX, and very few (not really enough) in windows.
Special thanks. We really want to thanks to: Of course, Stephane Ducasse and all ESUG people, for the interest (and funding) showed on this project. Norbert Sendetzky (OpenDBX author) because all the help he gave us not only with OpenDBX but also with SqueakDBX.
Help wanted. We are really convinced about open source projects. A software project is better when people share opinions, work, advice, comments, etc. We would really appreciate if you: Give us opinions, comments, ideas, new features, complaints and so on. tell us if you find a bug. tell us if you test SqueakDBX with other RDMBS or OS different from the ones we tested. Just to know if it works or not :)
Full documentation, installation and getting started instructions can be found at wiki page: http://wiki.squeak.org/squeak/6052
Packages can be installed from Universe (3.10) or SqueakMap. Current version is 0.7.1
Sources can be download from SqueakSource http://www.squeaksource.com/SqueakDBX (it requires FFI installed)
Many thanks, SqueakDBX team
Sources can be download from SqueakSource http://www.squeaksource.com/SqueakDBX http://www.squeaksource.com/SqueakDBX (it requires FFI installed)
Many thanks, SqueakDBX team
So by using FFI it locks the VM and serializes all the requests through a single pipe allowing a long running query to block all other queries essentially freezing the VM?
Ramon Leon http://onsmalltalk.com
Ramon Leon wrote:
Sources can be download from SqueakSource http://www.squeaksource.com/SqueakDBX http://www.squeaksource.com/SqueakDBX (it requires FFI installed)
Many thanks, SqueakDBX team
So by using FFI it locks the VM and serializes all the requests through a single pipe allowing a long running query to block all other queries essentially freezing the VM?
Depends on whether OpenDBX (or the driver) support asynchronous mode.
Cheers, - Andreas
So by using FFI it locks the VM and serializes all the
requests through a
single pipe allowing a long running query to block all other queries essentially freezing the VM?
Depends on whether OpenDBX (or the driver) support asynchronous mode.
Cheers,
- Andreas
But of course, hence the question mark!
Ramon Leon http://onsmalltalk.com
Hi, The driver uses asynchronous calls for comunicating with it's respective engines, as far as the engine allow it, that's one of the powerful features it has. This is how we think a query must be:
1. call to execute- an external call 2. wait for answer (a request to ask if there is already an answer, otherwise, yields for N time)- another external call 2.1. loop step 2 until there is an answer. 3. retrieve result rows, one by one -an external call each one
Of course, all of this calls are through FFI, and therefore, blocking... but is better to have many mini-blocks than just one large block.
That's the theory... right now, the fact is that very large operations on same connection are freezing the image... we are working on that :)
Cheers, Esteban
On 2008-08-20 15:09:17 -0300, "Ramon Leon" ramon.leon@allresnet.com said:
So by using FFI it locks the VM and serializes all the
requests through a
single pipe allowing a long running query to block all other queries essentially freezing the VM?
Depends on whether OpenDBX (or the driver) support asynchronous mode.
Cheers,
- Andreas
But of course, hence the question mark!
Ramon Leon http://onsmalltalk.com
But... I just tested this:
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
And it works perfect! No freeze is observed :)
And parallel operations seems to work too... so out "real-tiny-blocks" approuch seems to work.
Cheers, Esteban
On 2008-08-20 16:00:16 -0300, Esteban Lorenzano estebanlm@gmail.com said:
Hi, The driver uses asynchronous calls for comunicating with it's respective engines, as far as the engine allow it, that's one of the powerful features it has. This is how we think a query must be:
- call to execute- an external call
- wait for answer (a request to ask if there is already an answer,
otherwise, yields for N time)- another external call 2.1. loop step 2 until there is an answer. 3. retrieve result rows, one by one -an external call each one
Of course, all of this calls are through FFI, and therefore, blocking... but is better to have many mini-blocks than just one large block.
That's the theory... right now, the fact is that very large operations on same connection are freezing the image... we are working on that :)
Cheers, Esteban
On 2008-08-20 15:09:17 -0300, "Ramon Leon" ramon.leon@allresnet.com said:
So by using FFI it locks the VM and serializes all the
requests through a
single pipe allowing a long running query to block all other queries essentially freezing the VM?
Depends on whether OpenDBX (or the driver) support asynchronous mode.
Cheers,
- Andreas
But of course, hence the question mark!
Ramon Leon http://onsmalltalk.com
On 2008-08-20 16:16:00 -0300, Esteban Lorenzano estebanlm@gmail.com said:
But... I just tested this:
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
And it works perfect! No freeze is observed :)
And parallel operations seems to work too... so out "real-tiny-blocks" approuch seems to work.
I meant: our "real-tiny-blocks" approach :)
Cheers, Esteban
On 2008-08-20 16:00:16 -0300, Esteban Lorenzano estebanlm@gmail.com said:
Hi, The driver uses asynchronous calls for comunicating with it's respective engines, as far as the engine allow it, that's one of the powerful features it has. This is how we think a query must be:
- call to execute- an external call
- wait for answer (a request to ask if there is already an answer,
otherwise, yields for N time)- another external call 2.1. loop step 2 until there is an answer. 3. retrieve result rows, one by one -an external call each one
Of course, all of this calls are through FFI, and therefore, blocking... but is better to have many mini-blocks than just one large block.
That's the theory... right now, the fact is that very large operations on same connection are freezing the image... we are working on that :)
Cheers, Esteban
On 2008-08-20 15:09:17 -0300, "Ramon Leon" ramon.leon@allresnet.com said:
So by using FFI it locks the VM and serializes all the
requests through a
single pipe allowing a long running query to block all other queries essentially freezing the VM?
Depends on whether OpenDBX (or the driver) support asynchronous mode.
Cheers,
- Andreas
But of course, hence the question mark!
Ramon Leon http://onsmalltalk.com
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
And it works perfect! No freeze is observed :)
But will this work...
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
[ self executeAndRetreiveAShortQuery ] forkAt: self userBackgroundPriority.
And the short query finish first even though the longer one started first?
Ramon Leon http://onsmalltalk.com
He, I'm testing it now... in a while I upload results :)
On 2008-08-20 17:13:31 -0300, "Ramon Leon" ramon.leon@allresnet.com said:
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
And it works perfect! No freeze is observed :)
But will this work...
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
[ self executeAndRetreiveAShortQuery ] forkAt: self userBackgroundPriority.
And the short query finish first even though the longer one started first?
Ramon Leon http://onsmalltalk.com
Ok, new tiny-blocks tests:
I tested:
1) Inserting 100.000 rows into two tables. 2) Select 100.000 rows from a table.
First of all, running with background priority image isn't freezing, and FFI calls are not perceived. This is important because this allows me to still render seaside pages (for example) for another user, when there is one user making a big query. Second, I followed rows insertion (simple "select count(*) from bla1" in psql tool) and the insertions is happening in parallel.
Of course FFI calls are still happening, and there are many "micro-blocks", so the total time expended in each process is equal to the addition of all FFI blocks + squeak calculations. But is important to note that (2 parallel queries time) < (query1 time + query2 time)...
I'm going to put all this benckmarks in the wiki page (and the sources in squeaksource)
Cheers, Esteban
On 2008-08-20 17:17:13 -0300, Esteban Lorenzano estebanlm@gmail.com said:
He, I'm testing it now... in a while I upload results :)
On 2008-08-20 17:13:31 -0300, "Ramon Leon" ramon.leon@allresnet.com said:
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
And it works perfect! No freeze is observed :)
But will this work...
[ self executeAndRetreiveAVeryLongQuery ] forkAt: self userBackgroundPriority.
[ self executeAndRetreiveAShortQuery ] forkAt: self userBackgroundPriority.
And the short query finish first even though the longer one started first?
Ramon Leon http://onsmalltalk.com
squeak-dev@lists.squeakfoundation.org
-
Andreas Raab -
Esteban Lorenzano -
Ramon Leon