[xsd-users] Re: Embedded XSD for games...

[Boris Kolpackov]

Hi Sam,

I think xsd-users is a more appropriate place to discuss this. I've
CC'ed it to my reply.

I am not sure if you looked into this but from your description it
appear that the binary serialization of the in-memory representation
can do what you want. The idea of this feature is that you can
instruct XSD (see the --generate-insertion and --generate-extraction
options as well as the 'binary' example) to generate stream insertion
operators and extraction constructors. Pretty much any custom stream
type can be used. ACE CDR (CORBA over-the-wire format) is supported
out of the box. We've done some measurements and the resulting
representation is a lot smaller and faster to parse.

This would work like so: you will generate two version of the C++
code from your schema. The first is for the PC and will include XML
parsing and binary serialization code. The second is for the console
and will only include binary extraction code without any XML stuff.
The PC code will parse XML, serialize to binary format, and send
it to the console. The console will extract the data from the
binary format which can then be used by the application.

Let me know if I missed anything from your requirements.

hth,
-boris


[The original email follows for context.]

sam scott <billythefisherman at googlemail.com> writes:

> Hi Boris (no one else seems to post here!) - I hope I've got this post in
> the right place but that's really what I'm trying to find out.  I'm writing
> an article which I hope to post on a game development website which proposes
> an 'ideal' pipeline for getting all our data requirements through one
> conversion utility from the many different source applications we use.
>
> All of these source apps (Maya, Proprietary Editor, MS Office) output XML in
> varying formats and need converting to binary versions PC side - we're
> incredibly data heavy and so any processing of XML data on the console side
> is really out of bounds.  So at home I've started to experiment with your
> xsd lib which is fantastic it must be said but have found one large problem
> to do with my problem domain.
>
> The problem goes like this: to get binary data off of the PC to console i
> must deal with two problems: pointers and virtual table pointers.  The
> former is easy and the latter a little less so but none the less possible
> I actually use a lib from an article in Games Programming Games 5: "1.13 A
> Transparent Class Saving and Loading Trick".  This basically replaces all
> the pointers with a special 'smart pointer' which then gets turned into an
> offset and put into a table at save time (PC side) and remapped out to the
> correct memory address at load time (console side).
>
> This means that due to the autogenerated nature of XSD and XSDe,
> I have to add in an extra conversion step PC side to:
>
> a) turn all the pointers into freeze pointers
> b) strip down the data structures to there bare
> bones (most just need to be simple arrays and structures) - actually XSDe
> seems to provide a solution this point
> c) remove as much inheritance as possible (usually i don't want any as
> dealing with moving vtbl pointers over to target platform is a royal pain)
>
> This step is error prone and time consuming and goes against the reason of
> using an XML data binding step - the data binding helps but I feel I should
> be able to get to this format direct ie use the auto generated data
> structures console side)
>
> So (and we fianlly get to the meat and potatoes here) to solve all my
> problems I get the feeling I need to use your xsd frontend to write my own
> version of XSD which fills my requirements: do you think this is so or is
> there a method you've already devised of to deal with this type of
> situation?
>
> Thanks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 652 bytes
Desc: Digital signature
Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20070514/01bced63/attachment.pgp