You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a formalized ticket for this PR #4858 so people are aware and can contribute to figuring out if this idea makes sense... and if so then what needs to be done before this can be merged in from a feature requirement perspective.
Feature Description and Motivation
Metadata Override
Often safetensors provided by external parties maybe missing certain metadata or have incorrectly formatted metadata. To make things easier to find in hugging face, accurate metadata is a must.
The idea is to allow users to override metadata in the generated gguf by including a json metadata file
{
"general.name": "TinyLLama",
"general.version": "v0",
"general.author": "mofosyne",
"general.url": "https://huggingface.co/mofosyne/TinyLLama-v0-llamafile",
"general.description": "This gguf is ported from a first version of Maykeye attempt at recreating roneneldan/TinyStories-1M but using Llama architecture",
"general.license": "apache-2.0",
"general.source.url": "https://huggingface.co/Maykeye/TinyLLama-v0",
"general.source.huggingface.repository": "https://huggingface.co/Maykeye/TinyLLama-v0"
}
At the moment the PR will only recognize metadata that is explicitly defined in python gguf writer, so any that is not yet defined will be ignored. If you think that should not be the case, then definitely make your case and I'll may see how easy it is to just allow for user defined metadata in the metadata override json.
Default outfile name generation
To help promote consistent naming scheme I've created a --get-outfile and also adjusted the default file naming function to be based on this format <Model>-<Version>-<ExpertsCount>x<Parameters>-<Quantization>.gguf (detailed description in PR).
Also when generating a gguf, if you don't name your output file it will default to a file that may look similar to TinyLLama-v0-5M-F16.gguf.
This format is based on what I've generally observed in how people name their files in huggingface (using vibes, so if you think my naming scheme needs adjusting, let me know).
I've already merged in all the changes required to support this so the only changes that needs to be checked is convert.py (Other scripts may need to be adjusted to port over the file conventions and metadata... but just focusing on convert.py as that's the lowest hanging fruit and a good MVP to see if it makes sense in the real world). This should make it easier to review and then merge.
The text was updated successfully, but these errors were encountered:
Feature #4858 now added to convert.py . Feel free to reopen if it works out and we want to add this to the other converter scripts beyond safetensors to gguf
This is a formalized ticket for this PR #4858 so people are aware and can contribute to figuring out if this idea makes sense... and if so then what needs to be done before this can be merged in from a feature requirement perspective.
Feature Description and Motivation
Metadata Override
Often safetensors provided by external parties maybe missing certain metadata or have incorrectly formatted metadata. To make things easier to find in hugging face, accurate metadata is a must.
The idea is to allow users to override metadata in the generated gguf by including a json metadata file
where the metadata override file may look like:
At the moment the PR will only recognize metadata that is explicitly defined in python gguf writer, so any that is not yet defined will be ignored. If you think that should not be the case, then definitely make your case and I'll may see how easy it is to just allow for user defined metadata in the metadata override json.
Default outfile name generation
To help promote consistent naming scheme I've created a
--get-outfile
and also adjusted the default file naming function to be based on this format<Model>-<Version>-<ExpertsCount>x<Parameters>-<Quantization>.gguf
(detailed description in PR).So for example when you call this command
you would get
Also when generating a gguf, if you don't name your output file it will default to a file that may look similar to
TinyLLama-v0-5M-F16.gguf
.This format is based on what I've generally observed in how people name their files in huggingface (using vibes, so if you think my naming scheme needs adjusting, let me know).
Possible Implementation
I have already tested the overall flow when generating this via this bash script https://huggingface.co/mofosyne/TinyLLama-v0-5M-F16-llamafile/blob/main/llamafile-creation.sh using
convert.py
and we already have a PR #4858 waiting for merging.I've already merged in all the changes required to support this so the only changes that needs to be checked is
convert.py
(Other scripts may need to be adjusted to port over the file conventions and metadata... but just focusing on convert.py as that's the lowest hanging fruit and a good MVP to see if it makes sense in the real world). This should make it easier to review and then merge.The text was updated successfully, but these errors were encountered: