-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG2 #416
Labels
enhancement
New feature or request
Comments
Thanks for the suggestions, we welcome contributions from the community. |
@Phani1609 would it be something you are interested to add to W&B implementation here? https://github.com/meta-llama/llama-recipes/blob/main/src/llama_recipes/finetuning.py#L52 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
馃殌 The feature, motivation and pitch
I propose adding a feature to improve the visualization of loss curves in LLaMA-Recipes. Currently, the loss curves may exhibit spikes or irregularities, making it challenging to interpret training progress accurately. By enhancing the loss curve visualization, users can better analyze model training dynamics, identify potential issues, and make informed decisions for model optimization.
Motivation:
The motivation for this proposal stems from the need for clearer and more informative loss curve visualization in LLaMA-Recipes. Clearer visualization can aid researchers and practitioners in understanding model behavior, diagnosing training issues, and fine-tuning training strategies to improve model performance.
Pitch:
The proposed feature involves implementing smoother and more informative loss curve visualization techniques in LLaMA-Recipes. This could include methods such as moving average smoothing, adaptive smoothing algorithms, or interactive visualization tools that allow users to explore and analyze loss curves dynamically.
Alternatives
Manual post-processing of loss curve data to achieve smoother visualization.
Using external visualization libraries or tools to visualize loss curves outside of LLaMA-Recipes.
Additional context
No response
The text was updated successfully, but these errors were encountered: