v0.7.4

Release Note Finetuner 0.7.4

This release covers Finetuner version 0.7.4, including dependencies finetuner-api 0.5.5 and finetuner-core 0.13.0.

This release contains 2 new features, 3 refactoring, 1 bug fix, and 3 documentation improvements.

🆕 Features

Layer-wise learning rate decay (LLRD) (#697)

LLRD sets a large learning rate for the top (last) layer and uses a multiplicative decay rate to decrease the learning rate layer-by-layer from top (last) to bottom (first). With high learning rates, the features recognized by the top layers change more and adapt to new tasks more easily, while the bottom layers have low learning rates and more easily preserve the features learned during pre-training:

import finetuner
run = finetuner.fit(
    ...,
    optimizer='Adam'
+   optimizer_options={'layer_wise_lr_decay': 0.98},
    ...,
)

Support CSV for similarity-based training (#696)

In the previous release, Finetuner added support for training with data that is not specifically labeled, but each pair of training items has a numerical similarity score between 0.0 (totally dissimilar) and 1.0 (totally the same). This extends the potential scenarios for which Finetuner is applicable.

Now, users can prepare training data with a CSV file like this:

The weather is nice, The weather is beautiful, 0.9
The weather is nice, The weather is bad, 0

⚙ Refactoring

Unify supported backbone names (#700)

We unified backbone names into the name-size-[lang] format. Now model names are easier to understand and remember. For example: bert-base-cased becomes bert-base-en, openai/clip-vit-base-patch32 becomes clip-base-en, resnet152 becomes resnet-large. Users can see the support backbones with:

finetuner.describe_models(task='text-to-image')

Note that you can keep using the old names, for example:

# build a zero-shot resnet embedding model
model = finetuner.build_model('resnet50')

# this is identical to
model = finetuner.build_model('resnet-base')

Simplify model offerings. (#700)

To simplify our offerings and create a more intuitive user experience, we have decreased the number of supported machine-learning models. By doing so, we hope to make it easier for our users to navigate our platform and find the models that best suit their needs.

Change the default batch_size for EvaluationCallback from 8 to 32.

We recently implemented a change to our EvaluationCallback by increasing the batch size. This has resulted in a significant improvement in evaluation speed, which in turn has reduced the overall cost of evaluation. By processing data more efficiently, we can evaluate models more quickly and accurately, which ultimately benefits our users.

🐞 Bug Fixes

Fix IndexError in EvaluationCallback when 'gather_examples' is enabled.

We resolved an issue in the EvaluationCallback where the 'list out of range' error was occurring when 'gather_examples' was enabled. This fix ensures that the evaluation callback works correctly with 'gather_examples' and enables users to gather evaluation examples without encountering errors.

📗 Documentation Improvements

• Add section for layer-wise learning rate decay (LLRD) (#697)
• Add section for CosineSimilarityLoss (#697)
• Add section on creating query-document pairs as CSV (#697)

🤟 Contributors

We would like to thank all contributors to this release:

• Wang Bo (@bwanglzu)
• Louis Milliken (@LMMilliken)
• Michael Günther (@guenthermi)
• George Mastrapas (@gmastrapas)
• Scott Martens (@scott-martens)