Archive for the ‘ORM’ Category

ODB 1.6.0 released

Tuesday, October 4th, 2011

ODB 1.6.0 was released today.

In case you are not familiar with ODB, it is an object-relational mapping (ORM) system for C++. It allows you to persist C++ objects to a relational database without having to deal with tables, columns, or SQL, or manually writing any of the mapping code.

This version includes a large number of major new features, small improvements, and bug fixes. For an exhaustive list of changes, see the official ODB 1.6.0 release announcement. As usual, below I am going to examine the most notable new features in more detail.

Views

No doubt the biggest feature in this release is the introduction of the view concept. An ODB view is a C++ class that embodies a light-weight, read-only projection of one or more persistent objects or database tables or the result of a native SQL query execution.

Some of the common applications of views include loading a subset of data members from objects or columns from database tables, executing and handling results of arbitrary SQL queries, including aggregate queries, as well as joining multiple objects and/or database tables using object relationships or custom join conditions.

Many relational databases also define the concept of views. Note, however, that ODB views are not mapped to database views. Rather, by default, an ODB view is mapped to an SQL SELECT query. However, if desired, it is easy to create an ODB view that is based on a database view.

As an example, consider a simple person persistent class:

#pragma db object
class person
{
  ...
 
  #pragma db id auto
  unsigned long id_;
 
  std::string first_;
  std::string last_;
  unsigned short age_;
};

Let’s say we want to define a view that returns the number of people stored in our database:

#pragma db view object(person)
struct person_count
{
  #pragma db column("count(" + person::id_ + ")")
  std::size_t count;
};

And here is how we can use this view to get the total head count:

odb::result<person_count> r (db.query<person_count> ());
 
const person_count& c (*r.begin ()); // Exactly one element.
cout << c.count << endl;

Or we can count people that match only certain criteria. For example, here is how we can find out how many people in our database are younger than 30:

typedef odb::query<person_count> query;
typedef odb::result<person_count> result;
 
result r (db.query<person_count> (query::age < 30));
 
const person_count& c (*r.begin ());
cout << c.count << endl;

ODB views can be defined in terms of one or more persistent objects, database tables, a combination of the two, or as a native SQL query. As a result, there are a lot of different things that can be achieved with views. If you would like to learn more, refer to Chapter 9, “Views” in the ODB Manual. There is also the view example in the odb-examples package.

NULL Semantics

ODB now supports the so-called NULL semantics wrappers which allow us to transform any value type to a type that can have the special NULL state. We can use the standard smart pointers as well as the odb::nullable “optional” container as NULL wrappers. The Boost profile adds support for boost::shared_ptr and boost::optional while the Qt profile adds support for QSharedPointer. We can also use our own smart pointers or “optional” containers as NULL wrappers.

As an example, let’s say we would like to store the optional middle name in our person class from the previous section. Here is how we can do it using std::auto_ptr:

#pragma db object
class person
{
  ...
 
  std::string first_;
 
  #pragma db null
  std::auto_ptr<std::string> middle_;
 
  std::string last_;
};

Now, if we don’t want to incur a dynamic memory allocation just to get the NULL semantics, we can use the odb::nullable container instead:

#include <odb/nullable.hxx>
 
#pragma db object
class person
{
  ...
 
  std::string first_;
  odb::nullable<std::string> middle_;
  std::string last_;
};

Note that here we don’t need the db null pragma since odb::nullable enables NULL by default.

We could also use boost::optional instead of odb::nullable, provided we enable the Boost profile (-p boost ODB compiler option):

#include <boost/optional.hpp>
 
#pragma db object
class person
{
  ...
 
  std::string first_;
  boost::optional<std::string> middle_;
  std::string last_;
};

For more information on this feature, refer to Section 7.3, “NULL Value Semantics” in the ODB manual.

Erase Query

The new erase_query() function allows us to delete the database state of multiple objects matching certain criteria. It uses the same query expression as the query() function. For example, this is how we can delete all the people in our database that are younger than 30:

db.erase_query<person> (odb::query<person>::age < 30)

For more information on this feature, refer to Section 3.10, “Deleting Persistent Objects” in the ODB manual.

BLOB Handling

It is now possible to use the std::vector<char> type to store BLOB data in the database. Note, however, that to enable this mapping, we need to explicitly specify the database type, for example:

#pragma db object
class person
{
  ...
 
  #pragma db type("BLOB")
  std::vector<char> public_key_;
};

Alternatively, we can do it on the per-type basis, for example:

typedef std::vector<char> buffer;
#pragma db value(buffer) type("BLOB")
 
#pragma db object
class person
{
  ...
 
  buffer public_key_; // Mapped to BLOB.
};

Expressive Query Syntax

Prior to this release we used the scope resolution operator (::) when referring to members inside composite values and pointed-to objects in query expressions. For example:

db.query<person> (query::employer::name == "Example, Inc");

The problem with this approach is that it is impossible to say whether the member is inside a composite value or an object just by looking at the expression. In the above example, employee could be a composite value type or a pointer to an object. To make the queries more expressive, we have changed the syntax to use the member access operator (.) when referring to members inside composite value types and to use the member access operator via pointer (->) when referring to members inside related objects. As a result, the above query will look like this if employee is a composite value:

db.query<person> (query::employer.name == "Example, Inc");

And like this, if it is a pointer to an object:

db.query<person> (query::employer->name == "Example, Inc");

Other interesting new features in this release include the --table-prefix ODB compiler option, the odb::connection interface, and support for multiplexing several transactions on the same thread. For more information on these and other features, see the official ODB 1.6.0 release announcement.

ODB 1.5.0 released

Tuesday, July 26th, 2011

ODB 1.5.0 was released today.

In case you are not familiar with ODB, it is an object-relational mapping (ORM) system for C++. It allows you to persist C++ objects to a relational database without having to deal with tables, columns, or SQL, or manually writing any of the mapping code.

As usual, for the complete list of changes see the official ODB 1.5.0 announcement. However, to wet your appetite, the big new feature in this release is no doubt support for the PostgreSQL database, thanks to several months of hard work by Constantin. Below I am going to examine this and another new feature in more detail. There are also some performance numbers for dessert.

PostgreSQL support

Support for PostgreSQL is provided by the libodb-pgsql runtime library. All the standard ODB functionality is available to you when using PostgreSQL, including support for containers, object relationships, queries, date-time types in the Boost and Qt profiles, etc. In other words, this is complete, first-class support, similar to that provided for MySQL and SQLite. There are a few limitations, however, most of which are imposed by the underlying C API as defined by PostgreSQL’s libpq. Those are discussed in Chapter 13, “PostgreSQL Database” in the ODB Manual.

For connection management in PostgreSQL, ODB provides two standard connection factories (you can also provide your own if so desired): new_conection_factory, and conection_pool_factory.

The new connection factory creates a new connection whenever one is requested. Once the connection is no longer needed, it is closed.

The connection pool factory maintains a pool of connections and you can specify the min and max connection counts for each pool created. This factory is the default choice when creating a database instance.

If you had any prior experience with ODB, you are probably aware that one of our primary goals is high performance and low overhead. For that we use native database APIs and all the available performance enhancing features (e.g., prepared statements). We also cache connections, statements, and even memory buffers extensively. The PostgreSQL runtime is no exception in this regard. The question you are probably asking now is how does it stack up, performance-wise, against other databases that we support.

Well, the first benchmark that we tried is the one from the Performance of ODB vs C# ORMs post. Essentially we are measuring how fast we can load an object with a couple of dozen members from the database. It takes ODB with PostgreSQL 9.0.4 27ms per 500 iterations (54μs per object). For comparison, using MySQL 5.1.49 it takes 24ms (48μs per object) and SQLite 3.7.5 — 7ms (14μs per object). So PostgreSQL is more or less on par with MySQL here.

What was more surprising is the concurrent access performance. We have an update-heavy, highly-contentious multi-threaded test in the ODB test suite, the kind you run to make sure things work properly in multi-threaded applications (see odb-tests/common/threads if you are interested in details). It normally takes several minutes to complete and pushes my 2-CPU, 8-core Xeon E5520 machine, which runs the database server, close to 100% CPU utilization. The surprising part is that PostgreSQL 9.0.4 is more than 10 times faster on this test than MySQL 5.1.49 with the InnoDB backend (186s for MySQL, 48s for SQLite, and 12s for PostgreSQL). Postgres developers seem to be doing something right.

Let me also note that these numbers should be taken as indications only. It is futile to try to extrapolate some benchmark results to your application when it comes to databases. The only reliable approach is to create a custom test that mimics your application’s data, concurrency, and access patterns. Luckily, with ODB creating such a test is a very easy job.

Database operations callbacks

Another new feature in this release is support for per-class database operations callbacks. Now a persistent class can register a callback function that will be called before and after every database operation (such as persist, load, update, or erase) is performed on an object of this class. For example, we can use a callback to re-calculate some transient values based on the data retrieved from the database after the load operation:

#pragma db object callback(init)
class person
{
  ...
 
  date born_;
 
  #pragma db transient
  unsigned short age_;
 
  void
  init (odb::callback_event e, odb::database&)
  {
    switch (e)
    {
    case odb::callback_event::post_load:
    {
      // Calculate age from the date of birth.
      ...
      break;
    }
    default:
      break;
    }
  }
};

As shown in the above example, a database operations callback can be used to implement object-specific pre and post initializations, registrations, and cleanups. For more information on this feature, refer to Section 10.1.4, “Callback” in the ODB Manual.

See you at BoostCon 2011

Tuesday, May 3rd, 2011

I am giving two talks at this year’s BoostCon conference. The first is titled “Parsing C++ with GCC plugins”. It is about the new GCC plugin architecture and how it can be used for parsing C++.

The second talk is titled “Object-Relational Mapping with ODB and Boost”. It covers the ODB object-relational mapping (ORM) system for C++ and its integration with Boost.

If any of you are attending the conference, please say Hi!