[xsd-users] Tree streaming serialization

[Pavel Fiala]

Hi Boris,

I executed the streaming example with 1M and 10M position elements. The 
output file size is ~38 MB for 1M position elements and ~387 MB for 10M 
position elements, the memory consumption for 10M is basically 10 times 
1M memory, plots are attached.

If you have anything new about serialization, I would be interested in 
it as the streaming serialization is the main issue with my own schema.

Thank you,
Pavel

On 9.2.2014 11:28, Boris Kolpackov wrote:
> Hi Pavel,
>
> Pavel Fiala <pavel.fiala at cern.ch> writes:
>
>> I'm trying to use XSD/Tree streaming serialization of a large
>> document but I'm facing a problem - memory consumption. It's
>> possible to reproduce it with the bundled example
>> "xsd/examples/cxx/tree/streaming". I've changed the number of
>> generated position elements to 50000 and it takes ~9 MB of memory,
>> see attached screenshot.
> What happens if you try to serialize, say, 1M of position elements
> (maybe send them to /dev/null)? The 9MB is not really large enough
> to indicate constantly increaing memory consumption (it could just
> be cached; see below).
>
>
>> It seems that the problem is how DOMElements are created as they are
>> probably not released.
> What happens with elements is they are returned to the document's
> cache and then (potentially) reused.
>
> BTW, for the next release of XSD we have improved and generalized
> streaming parser and serializer significantly. In particular, you
> no longer need to modify them for your XML vocabulary. They are
> completely generic and can do streaming parsing/serialization
> any number of levels deep. Let me know if you would like the
> updated 'streaming' example.
>
> Boris

-------------- next part --------------
A non-text attachment was scrubbed...
Name: position-1M.png
Type: image/png
Size: 35118 bytes
Desc: not available
Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20140210/3ad96685/position-1M-0001.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: position-10M.png
Type: image/png
Size: 34194 bytes
Desc: not available
Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20140210/3ad96685/position-10M-0001.png