[odb-users] Questions and future direction of ODB

[Raindog]

On 10/19/2010 6:50 AM, Boris Kolpackov wrote:
> Hi,
>
> Raindog<raindog at macrohmasheen.com>  writes:
>
> >  Windows support is being consistently improved. In fact, it compiles
> >  very easy out of the box now on windows.
>
> That's good to know, thanks. Just to clarify, is this just the compiler
> frontend or the whole toolchain? If it is the toolchain, does it reuse
> some existing bits, e.g., from MinGW?
>    
I'm not exactly sure about that to be honest. I seem to recall them not 
having their own library implementation yet, at least not a complete 
one, and IMO, it would serve them well just to use GCC's
>
> >  Sorry, I meant something like a use case for #2. If I have a query or
> >  stored procedure that is not just pulling single entities from a
> >  database, IE i have several joins, etc, and I want to display these
> >  results in something like an HTML table, it seems like the current
> >  feature set makes that complex to do.
>
> The "object-oriented database way" of doing it would be to pull individual
> objects into the application's memory and then intersect them there using
> C++. The advantage of this approach is that it may reduce the database
> load since the processing will be done by the application instead of the
> database server. I said "may" because it is also possible that this will
> increase the database load if the number of returned objects is large.
>    
I can see how in a website for example, it might reduce load on the DB 
by offsetting it to the front-end, which in general is easier to scale 
than a DB, however for general purpose data processing, this can lead to 
orders of magnitude slower performance due to the latency in retrieving 
data from the database, just think of how slow this approach can be if 
the database is not on the same machine as the application making the DB 
requests.

> But it seems that this is not a very popular approach (you are the second
> person who asks for support for joins). So we will need to think of a way
> to support this. Maybe something like a read-only, "view object":
>
> #pragma view table("table1", "table2")
> class test_view
> {
>    #pragma column("table1.v1")
>    string v1;
>
>    #pragma column("table2.v2")
>    string v2;
> };
>
> db->query<test_view>  (query::v1 == query::v2);
>
> What do you think?
>    
I think defining a structure with which one can store results is 
perfectly acceptable, although many times I think that just using a 
Boost.Tuple is appropriate. Where something like this:

       result r ("select a,b,c from t");

       for (result::iterator i (r.begin ()); i != r.end (); ++i)
       {
         cout << "a: " << boost::at<0>(*i) << " b: " boost::at<1>(*i) << 
" c: " << boost::at<2>(*i) << endl;
       }

Where the result::iterator dereferences to, for example, a 
boost::tuple<x,x,x>;

Also, I can understand not wanting to take external dependencies where 
possible, but I think when it makes sense, taking a dependency on boost 
can save a lot of headache if you are instead faced with implementing 
something already provided by boost. You may otherwise get in same 
scenario that STL programmers face when trying to use MFC.



>
> >  If you are being bound to valid syntax of C++, isn't it possible to
> >  build a DSL on top of something like say, boost.proto similar to how
> >  boost.spirit is a great way for EBNF in C++
>
> Yes, but which functionality will this provide that is not already
> possible with the ODB Query Language? Can you give an example?
>    
My suggestion there was not about providing additional features to ODB 
QL, but rather suggested that the compiler plugin might have been 
unneeded altogether.

> Plus, Spirit is nice on toy languages. When you need to handle something
> non-trivial, things get very complicated very fast (I used Spirit to
> create a compiler for CORBA IDL).
>    
Someone wrote C++ parser called scalpel with Spirit. They recently 
abandoned the project however due to inability to compete with Clang due 
to the project being only 1 person, and Recently Joel wrote a lisp or 
scheme compiler in Spirit.
>
> >  If you already generate code, what harm is there in making it a little
> >  easier to use by making it harder to forget "boilerplate" code.
>
> I am not sure I follow you here. What is important to remember is that
> the friend declaration is there not for the ODB compiler but for the
> native C++ compiler that will be used to compile the generated code.
> So here we would have to modify the hand-written class declarations,
> not the generated code.
>
>    
Right. Which is exactly the point I am saying is not needed. If you are 
generating code, your code generator can easily determine that you are 
missing the include, or missing the friend declaration. The fact that it 
is needed by the generated code is an implementation detail that the 
client of ODB should not need to worry about.

> >  Ok, that makes sense, I've just been reading the documentation without
> >  looking at any of the source to ODB yet.
>
> Ok, I guess we need to clarify this in the documentation. Added to the
> TODO.
>
>
> >  Yes, but most support returning several result sets at once.
>
> Ok, I will need to look into this in more detail for other databases,
> but MySQL definitely doesn't support this.
>    
In MS SQL a stored procedure can return several result sets. I think 
that pgsql has the same ability.
>
> >  Yes. This way the user can receive some form of compile time
> >  verification that his code matches his schema, IE column names, table
> >  names, etc all match.
>
> Hm, I see how this can be useful but I don't see how this can be
> implemented in a manageable way. Ideally the ODB compiler would do
> this checking when compiling the header. Automatically connecting
> the database doesn't sound like a good idea. Perhaps we could use
> a DDL file that was dumped by the database to perform this
> verification. Would this meet your requirements?
>    
I think so, but IMO that would be more complex to parse than just 
connecting to the database as you will be getting a lot more data than 
needed, plus, each DB implementation will have slightly different 
formats. I think an alternative is to connect to the database, gather 
only the data you need and store it for later use. IMO lacking this 
means to validate ones code would prevent a lot of people from using ODB 
as it could potentially become just too complicated to debug all your 
table/column name mismatches and in cases where you do not have 100% 
code coverage or testing, you might find out about these errors days or 
weeks down the line.


Maybe I am wrong, but it seems you are really limiting the scope, and 
therefore overall utility of ODB by not automating as much as possible. 
IMO what the C++ community is lacking is something that has 
feature-parity of something like Hibernate, LINQ, Entity Framework or 
ActiveRecord. Data access in C++ is still more complex today than it was 
in C# 1.0 when all you had was bare bones ADO.NET. I envision your 
approach with ODB as the way to give C++ programmers a path towards 
being able to easily solve the kinds of problems that ActiveRecord and 
Hibernate are able to solve, but at this point, C++ programmers can't 
even begin to think about something like ActiveRecord++ because there 
still isn't any solid foundation upon which one can build.

> Boris
>
>