Archive for August, 2007

Writing XDR data to an expanding buffer

Monday, August 13th, 2007

The other day I was implementing support for XDR insertion/extraction in the C++/Tree mapping. XDR is a binary representation format that allows you to store, move, and then extract your data without worrying about word sizes (32 vs 64 bit), endian-ness, etc. XDR is available out of the box on pretty much every UNIX and GNU/Linux system as part of Sun RPC.

To test the performance of my implementation I was first serializing a large object model to a memory buffer and then deserializing it from that buffer. You can easily create an XDR stream to read/write the data from/to a fixed-size memory buffer (xdrmem_create) or a standard I/O stream (xdrstdio_create). There is also the xdrrec_create function which supports a record-oriented serialization as well as an abstract, callback-based underlying buffer management. This function is the only option for serializing to a dynamically-expanding buffer short of creating your own XDR stream. Unfortunately there aren’t many examples that show how to use it so I had to figure out the correct usage myself.

The following code fragment shows how to read/write XDR data using the xdrrec_create function. We use std::vector<char> as a buffer:

#include <vector>
#include <cstring> // std::memcpy
#include <iostream>
#include <rpc/xdr.h>
using namespace std;
typedef vector<char> buffer;
extern "C" int
overflow (char* p, char* data, int n)
  buffer* buf (reinterpret_cast<buffer*> (p));
  size_t size (buf->size ());
  buf->resize (size + n);
  memcpy (buf->data () + size, data, n);
  return n;
struct underflow_info
  buffer* buf;
  size_t pos;
extern "C" int
underflow (char* p, char* data, int n)
  underflow_info* ui (reinterpret_cast<underflow_info*> (p));
  size_t size (ui->buf->size () - ui->pos);
  n = size > n ? n : size;
  memcpy (data, ui->buf->data () + ui->pos, n);
  ui->pos += n;
  return n;
main ()
  buffer buf;
  // Serialize.
  XDR oxdr;
  xdrrec_create (&oxdr,
                 reinterpret_cast<char*> (&buf),
  oxdr.x_op = XDR_ENCODE;
  unsigned int i (10);
  xdr_uint32_t (&oxdr, &i);
  xdrrec_endofrecord (&oxdr, true); // flush the data.
  xdr_destroy (&oxdr);
  cerr << "size: " << buf.size () << endl;
  // Deserialize.
  underflow_info ui;
  ui.buf = &buf;
  ui.pos = 0;
  XDR ixdr;
  xdrrec_create (&ixdr,
                 reinterpret_cast<char*> (&ui),
  ixdr.x_op = XDR_DECODE;
  xdrrec_skiprecord (&ixdr);
  i = 0;
  xdr_uint32_t (&ixdr, &i);
  xdr_destroy (&ixdr);
  cerr << "i: " << i << endl;

The most non-obvious part in this code is the call to xdrrec_skiprecord.