2
Reply

Binary serilization of very large objects

Wojciech Szabowicz

Wojciech Szabowicz

Oct 19 2015 7:01 AM
1.3k
Greetings I'm having a problem and mayby someone will be able to aid me.
 
I'm trying to serilize a big dataset object, structure is like one object got 4 properties and those 4 properties has lots of other properties as objects and collections inside it. 
 
it looks like this: 
   [Serializable]     
   public class BackupProject : ISerializable
   {
public DocumentSet Documents;
       public CompareMetaDataSet MetaData;
       public RelationshipSet Relationships;
       public Dictionary<string, ItemConfig> DocumentData;

       public
BackupProject()
       {
       }

       public BackupProject(SerializationInfo objectInfo, StreamingContext objectContext)
      {
            Documents = (DocumentSet)objectInfo.GetValue("Documents", typeof(DocumentSet));
            MetaData = (CompareMetaDataSet)objectInfo.GetValue("MetaData", typeof(CompareMetaDataSet));
            Relationships = (RelationshipSet)objectInfo.GetValue("Relationships", typeof(RelationshipSet));
            DocumentData = (Dictionary<string, ItemConfig>)objectInfo.GetValue("DocumentData", typeof(Dictionary<string, ItemConfig>));
       }
            public void GetObjectData(SerializationInfo objectInfo, StreamingContext objectContext)
      {            
         objectInfo.AddValue("Documents", Documents);
         objectInfo.AddValue("MetaData", MetaData);
         objectInfo.AddValue("Relationships", Relationships);
         objectInfo.AddValue("DocumentData", DocumentData);
      }
 
It all comes to this that object is very large with data, now  I want to serilize using binary serilization 
 
   using (FileStream stream = File.Open(fullPath + "/" + backupFile, FileMode.Create))                 
   {
      var bformatter = new BinaryFormatter();
      using
(ZipOutputStream zipStream = new ZipOutputStream(stream))
      {
         zipStream.SetLevel(9);
         ZipEntry
zipEntry = new ZipEntry("BackupProject") {DateTime = DateTime.Now};
         zipStream.PutNextEntry(zipEntry);
         bformatter.Serialize(zipStream, documents);
      }
   }
 
But using standart binary methond and eaven with zip compression im getting :
 
The internal array cannot expand to greater than Int32.MaxValue elements.
Now is there a way to chunk up such an object using ISeriaziable and do not cross int32 max value ? 

Answers (2)