Serializer overloads for DataProvider duals of Tables

[Astn]

The Problem

Before I can serialize, I must create the objects to be serialized. In many cases these objects must be heap allocated classes. So this is an attempt at improving use cases like parsing a file, or receiving data in another format before getting it into a buffer.

A Solution

Generate a data provider interface of Tables that can be used in the serialization process. In my example I'm using a struct that contains a row index as its first parameter, and a function that would let it retrieve data for that row. This could be done in a variety of ways. But the basic idea is pass either pass a closure, or tuple (params, func) and let the serializer obtain the data directly without it being stored in a temporary object.

Some data

• Class1000 creates 1000 objects (heap) representing rows with around 100 cells (class) each.
• Struct1000 see Struct discussion Serializer overloads for Struct duals of Tables #211
• DataProvider1000 creates 1000 objects (struct) that have the data needed to fetch other needed data.

|           Method |      Mean |     Error |    StdDev |    StdErr |       Min |       Max |        Q1 |    Median |        Q3 |   Op/s |      Gen 0 |     Gen 1 | Gen 2 |    Allocated |
|----------------- |----------:|----------:|----------:|----------:|----------:|----------:|----------:|----------:|----------:|-------:|-----------:|----------:|------:|-------------:|
|        Class1000 | 29.386 ms | 0.5695 ms | 0.7796 ms | 0.1529 ms | 28.495 ms | 31.349 ms | 28.699 ms | 29.122 ms | 30.024 ms |  34.03 | 24031.2500 | 3937.5000 |     - | 147251.43 KB |
|       Struct1000 |  3.048 ms | 0.0607 ms | 0.1079 ms | 0.0171 ms |  2.917 ms |  3.310 ms |  2.944 ms |  3.025 ms |  3.127 ms | 328.13 |     3.9063 |         - |     - |     25.15 KB |
| DataProvider1000 |  1.408 ms | 0.0259 ms | 0.0419 ms | 0.0072 ms |  1.360 ms |  1.508 ms |  1.382 ms |  1.394 ms |  1.410 ms | 710.17 |    17.5781 |         - |     - |    118.77 KB |

Sample Benchmark

[Benchmark]
        public void Class1000()
        {
            var kvs = new List<KeyValue>();
            var map = new Map
            {
                Items = kvs
            };
            var buffer = new byte[1024];
            var now = DateTime.Now.ToFileTimeUtc();
            var rowsCount = rows.Length;
            var headerCount = header.Length;
            for (int r = 0; r < 1000; r++)
            {
                var row = rows[r % rowsCount];
                for (int col = 0; col < row.Length && col < headerCount; col++)
                {
                    KeyValue kv = new KeyValue { Key = header[col], Value = row[col], Timestamp = now };
                    if (!string.IsNullOrWhiteSpace(kv.Value.String))
                    {
                        kvs.Add(kv);
                    }
                }
                // serialize Kvs
                //var writtenSize = serialize(map, buffer);
                
                kvs.Clear();
            }
        }
        
        [Benchmark]
        public void DataProvider1000()
        {
            var kvs = new List<(int,Func<int,(string,string,long)>)>();
            var buffer = new byte[1024];
            var now = DateTime.Now.ToFileTimeUtc();
            var rowsCount = rows.Length;
            var headerCount = header.Length;
            for (int r = 0; r < 1000; r++)
            {
                var row = rows[r % rowsCount];
                for (int col = 0; col < row.Length && col < headerCount; col++)
                {
                    var f = new Func<int,(string,string,long)>((i) =>
                    {
                        return (Key: header[i], Value: row[i], Timestamp: now);
                    });
                    //var kv = (Key : header[col], Value : row[col], Timestamp : now );
                    kvs.Add((col, f));
                }
                // serialize Kvs
                //var writtenSize = serialize(map, buffer);
                
                kvs.Clear();
            }
        }