How to Load Data with Python Generators for Multi-Modal and Multi-Task

[daraghimo]

I would like to thank the authors and contributors of this wonderful AutoML system.
I am not a master of tensorflow and I depend on your tutorial and documentation.
The tutorial here show the way to load x,y data with python generators to a StructuredDataClassifier. What about a Customized Search Space case like the following example in the tutorial here

input_node1 = ak.ImageInput()
output_node = ak.Normalization()(input_node1)
output_node = ak.ImageAugmentation()(output_node)
output_node1 = ak.ConvBlock()(output_node)
output_node2 = ak.ResNetBlock(version="v2")(output_node)
output_node1 = ak.Merge()([output_node1, output_node2])

input_node2 = ak.StructuredDataInput()
output_node = ak.CategoricalToNumerical()(input_node2)
output_node2 = ak.DenseBlock()(output_node)

output_node = ak.Merge()([output_node1, output_node2])
output_node1 = ak.ClassificationHead()(output_node)
output_node2 = ak.RegressionHead()(output_node)

auto_model = ak.AutoModel(
    inputs=[input_node1, input_node2],
    outputs=[output_node1, output_node2],
    overwrite=True,
    max_trials=2,
)

image_data = np.random.rand(num_instances, 32, 32, 3).astype(np.float32)
structured_data = np.random.rand(num_instances, 20).astype(np.float32)
regression_target = np.random.rand(num_instances, 1).astype(np.float32)
classification_target = np.random.randint(5, size=num_instances)

auto_model.fit(
    [image_data, structured_data],
    [classification_target, regression_target],
    batch_size=32,
    epochs=3,
)

Can we load two input data with python generators and process each branch of them separately?
Can you transform this example to use generators for data that does not fit into memory ?
Thank you.

[daraghimo]

After searching and trying I reached the answer. I will put it here so it may help someone. I suggest adding it to the tutorial :).
here is an update to the generator displayed in the tutorial here If you have multiple inputs x and s:

def get_data_generator(n_batches, batch_size, n_features): # pass all variables used in generator
    def data_generator():
     # get x,s and y from your own loop
                    yield ((x,s), y)
    return data_generator
dataset_train = tf.data.Dataset.from_generator(
    get_data_generator(N_BATCHES, BATCH_SIZE, N_FEATURES), # pass training generator variables
    output_types=((tf.float32, tf.float32),tf.float32),   
    output_shapes= (([HEIGHT,WIDTH],[S_SHAPE]),[CLASSES]), 
).batch(BATCH_SIZE)
dataset_valid = tf.data.Dataset.from_generator(
    get_data_generator(N_BATCHES, BATCH_SIZE, N_FEATURES),  # pass validation generator variables
    output_types=((tf.float32, tf.float32),tf.float32),
    output_shapes= (([HEIGHT,WIDTH],[S_SHAPE]),[CLASSES]), 
).batch(BATCH_SIZE)

input_node_1 = ak.Input()
output_node_1 = ak.ConvBlock()(input_node_1)
input_node_2= ak.Input()
output_node_2 = ak.DenseBlock()(input_node_2)
merge_node= ak.Merge()([output_node_1, output_node_2])
output_node= ak.DenseBlock()(merge_node)
regression_output = ak.RegressionHead(loss=tf.losses.mean_squared_error,metrics=['mae'])(output_node)
auto_model= ak.AutoModel(  inputs=[input_node_1,input_node_2], outputs=[regression_output], max_trials=1,overwrite=True)
auto_model.fit(x=dataset_train,validation_data=dataset_valid)